Tower deformation detection method and device based on unmanned aerial vehicle and medium

Abstract

The application provides a tower shape change detection method and device based on a UAV, and a medium; the method comprises the following steps: acquiring a tower image through a UAV, and extracting global features and local features of the tower image; identifying key features through frequency channel attention and frequency space attention on the global features and the local features; performing an inflation convolution operation with different inflation rates on the key features to obtain multi-scale features; performing supervised learning on the multi-scale features to integrate semantic information and geometric information, and generating supervised attention features; decoding the supervised attention features to generate a change segmentation map, and performing target detection on the change segmentation map to obtain a tower shape change detection result; the method realizes efficient, accurate and automatic tower shape change detection, and does not require a detector to climb for detection, thereby improving safety.

Description

Technical Field

[0001] This application relates to the field of image processing, and more particularly to a method, device, and medium for detecting pole deformation based on unmanned aerial vehicles (UAVs). Background Technology

[0002] Power poles are supporting structures in power or communication systems, typically made of steel or concrete, used to erect high-voltage transmission lines, communication cables, or signal equipment. Their primary function is to support conductors, maintain safe distances, and ensure stable power or signal transmission. Natural disasters can easily damage power poles, causing deformation. Even minor deformations can disrupt their stress balance, reduce structural stability, and lead to fatigue damage to tower materials, loosening of connecting components, and even tower collapse. Simultaneously, minor deformations can alter the stress distribution on conductors, affecting safe distances for electrical equipment, increasing the risk of conductor breakage, equipment failure, and line tripping, threatening the continuity and reliability of power transmission. Therefore, post-disaster inspection and assessment of power pole deformation are necessary. Traditional manual inspections require inspectors to climb the poles, posing safety hazards, and are time-consuming and labor-intensive. Summary of the Invention

[0003] The following is an overview of the subject matter described in detail herein. This overview is not intended to limit the scope of the claims.

[0004] The purpose of this application is to at least partially solve one of the technical problems existing in the related technologies. The embodiments of this application provide a method, equipment and medium for detecting pole deformation based on UAV, which realizes efficient, accurate and automated detection.

[0005] An embodiment of the first aspect of this application provides a method for detecting pole deformation based on a drone, comprising:

[0006] Images of poles and towers are acquired by drones and input into the detection network. The images of poles and towers include multiple images of the same pole and tower at different times.

[0007] Extract the global and local features of the tower image;

[0008] Key features are identified from the global and local features using frequency channel attention and frequency spatial attention.

[0009] Multi-scale features are obtained by performing dilation convolution operations with different dilation rates on the key features;

[0010] Supervised learning is performed based on the multi-scale features to integrate semantic and geometric information and generate supervised attention features.

[0011] The supervised attention features are decoded to generate a change segmentation map, and target detection is performed based on the change segmentation map to obtain the tower deformation detection result.

[0012] According to certain embodiments of the first aspect of this application, the flight path of the UAV is planned according to the following steps:

[0013] Initialize a population, which includes multiple agents, each agent consisting of multiple individuals;

[0014] Calculate the fitness value of individuals in the population;

[0015] Update the frost layer factor and exploration factor based on the current iteration number;

[0016] When the frost factor is less than the preset adhesion coefficient, the agent's position is moved toward the current optimal solution;

[0017] When the frost factor is greater than or equal to the preset adhesion coefficient, the agent's position is updated by random perturbation according to the relationship between the fitness value and the exploration factor.

[0018] Randomly pair individuals in the population, perform arithmetic crossover on the paired individuals to generate new individuals, and calculate the position of the new individuals;

[0019] Replace individuals based on their fitness values ​​before and after the update;

[0020] Output the optimal path based on the location of the replaced individual.

[0021] According to certain embodiments of the first aspect of this application, updating the agent's position via random perturbation includes:

[0022] The agent's position is updated using the following formula: R(t) = R opt +(R avg -R prev )·L(Dim)·f(t); where,

[0023] In the formula, R(t) represents the position of the agent in the t-th iteration, R opt For the best location of the agent, R avg R is the average value of the optimal location. prev Let L(Dim) be the previous best position of the agent, f(t) be the Lévy flight function, f(t) be the transition factor function, c be a random number between [0,1], ζ be a random number between [0,1], T be the total number of iterations, s and β be constants, σ ​​be the factor function, μ be a random number between [0,1], and v be a random number between [0,1].

[0024] According to certain embodiments of the first aspect of this application, extracting global features from the tower image includes: extracting global features using a lightweight Transformer network.

[0025] According to certain embodiments of the first aspect of this application, extracting local features of the tower image includes: extracting local features using a lightweight attention inversion module;

[0026] The lightweight attention inversion module includes a local perception module, a lightweight multi-head attention module, and an inverted residual feedforward network.

[0027] In the local perception module, the input of the local perception module is added to the result of deep convolution, and the result is normalized by the layer to obtain the output of the local perception module. The output of the local perception module is used as the input of the lightweight multi-head attention module.

[0028] In the lightweight multi-head attention module, the first linear mapping result of the input of the lightweight multi-head attention module after linear mapping is used as the query of the multi-head attention mechanism; the second linear mapping result of the input of the lightweight multi-head attention module after depthwise convolution and linear mapping is used as the key of the multi-head attention mechanism; the third linear mapping result of the input of the lightweight multi-head attention module after depthwise convolution and linear mapping is used as the value of the multi-head attention mechanism; attention features are obtained through the multi-head attention mechanism based on the query, key, and value; the attention features are normalized by layers to obtain the output of the lightweight multi-head attention module; and the output of the lightweight multi-head attention module is used as the input of the inverted residual feedforward network.

[0029] In the inverted residual feedforward network, the input of the inverted residual feedforward network is convolved through multiple convolutional layers to obtain the output of the inverted residual feedforward network.

[0030] According to certain embodiments of the first aspect of this application, the identification of key features from the global features and the local features through frequency channel attention and frequency spatial attention includes:

[0031] The first aggregated feature is obtained by aggregating through global pooling along the channel dimension.

[0032] Separate the response components of different channels to key regions in the aggregated features;

[0033] A first low-frequency feature and a first high-frequency feature are extracted based on the response components, wherein the frequency of the first low-frequency feature is less than the frequency of the first high-frequency feature;

[0034] Convolve the first low-frequency feature and the first high-frequency feature to obtain the channel weights for each channel;

[0035] The second aggregated feature is obtained by aggregating the features in the spatial dimension through global pooling.

[0036] The second aggregated feature is convolved and residual connected to obtain the location information;

[0037] The second aggregated feature is subjected to fast Fourier transform and inverse Fourier transform to obtain the second low-frequency feature and the second high-frequency feature, wherein the frequency of the second low-frequency feature is less than the frequency of the second high-frequency feature.

[0038] Convolve the second low-frequency feature and the second high-frequency feature to obtain the spatial weights of each space;

[0039] Key features are identified based on the channel weights and spatial weights.

[0040] According to certain embodiments of the first aspect of this application, the dilated convolution operation includes:

[0041] The input to the dilated convolutional layer is subjected to average pooling, grouped convolution, and classification to obtain classification features;

[0042] Grouped convolutional features are obtained by performing grouped convolution on the input of the dilated convolutional layer.

[0043] The result of multiplying the classification features and the grouped convolutional features is reshaped and convolved to obtain the output of the dilated convolutional layer.

[0044] According to certain embodiments of the first aspect of this application, the method further includes:

[0045] The loss function is obtained based on the tower deformation detection results;

[0046] The parameters of the detection network are adjusted according to the loss function;

[0047] The loss function of the detection network is the sum of the classification loss and the weighted cross-entropy loss.

[0048] According to a second aspect of this application, an electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the UAV-based pole deformation detection method as described in the first aspect of this application.

[0049] According to a second aspect of this application, a computer storage medium stores computer-executable instructions for performing the UAV-based pole deformation detection method as described in the first aspect of this application.

[0050] The above scheme has at least the following beneficial effects: It acquires pole images using drones and extracts global and local features from these images; it identifies key features from the global and local features through frequency channel attention and frequency spatial attention; it performs dilation convolution operations with different dilation rates on the key features to obtain multi-scale features; it performs supervised learning based on the multi-scale features to integrate semantic and geometric information, generating supervised attention features; it decodes the supervised attention features to generate a change segmentation map, and performs target detection based on the change segmentation map to obtain the pole deformation detection result; it achieves efficient, accurate, and automated pole deformation detection, eliminating the need for inspectors to climb for inspection and improving safety. Attached Figure Description

[0051] The accompanying drawings are used to provide a further understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.

[0052] Figure 1 This is a flowchart illustrating the steps of the tower deformation detection method;

[0053] Figure 2 It is a step-by-step diagram for planning the flight path of a drone;

[0054] Figure 3 This is a sub-step diagram of step S400;

[0055] Figure 4 This is a diagram of the sub-steps of the dilated convolution operation;

[0056] Figure 5 This is a diagram of the detection network structure;

[0057] Figure 6 This is a structural diagram of the lightweight attention inversion module;

[0058] Figure 7 The process involves gradually modifying the structure diagram of the detection module;

[0059] Figure 8 This is a structural diagram of the supervised learning module. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0061] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, or the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0062] The embodiments of this application will be further described below with reference to the accompanying drawings.

[0063] The embodiments of this application provide a method for detecting pole deformation based on unmanned aerial vehicles (UAVs).

[0064] Reference Figure 1 The tower deformation detection method includes the following steps:

[0065] Step S200: Acquire pole images using a drone and input the pole images into the detection network;

[0066] Step S300: Extract global and local features from the tower image;

[0067] Step S400: Identify key features by using frequency channel attention and frequency spatial attention to identify global and local features;

[0068] Step S500: Perform dilation convolution operations with different dilation rates on the key features to obtain multi-scale features.

[0069] Step S600: Supervised learning is performed based on multi-scale features to integrate semantic and geometric information and generate supervised attention features;

[0070] Step S700: Decode the supervised attention features to generate a change segmentation map, and perform target detection based on the change segmentation map to obtain the tower deformation detection result.

[0071] It achieves efficient, accurate and automated detection of tower deformation, eliminating the need for inspectors to climb and improving safety.

[0072] The drone flew along a planned flight path and captured images of the towers along the way.

[0073] Reference Figure 2 Plan the drone's flight path by following these steps:

[0074] Step S110: Initialize the population;

[0075] Step S120: Calculate the fitness value of individuals in the population;

[0076] Step S130: Update the frost layer factor and exploration factor based on the current iteration number;

[0077] Step S140: When the frost factor is less than the preset adhesion coefficient, move the agent's position toward the current optimal solution;

[0078] Step S150: When the frost factor is greater than or equal to the preset adhesion coefficient, the agent's position is updated by random perturbation according to the relationship between the fitness value and the exploration factor.

[0079] Step S160: Randomly pair individuals in the population, perform arithmetic crossover on the paired individuals to generate new individuals, and calculate the position of the new individuals;

[0080] Step S170: Replace the individual based on its fitness value before and after the update;

[0081] Step S180: Output the optimal path based on the position of the replaced individual.

[0082] In UAV path planning, the initial swarm represents the possible flight paths of the UAV. Diverse paths can be generated in the initial stage of path planning to ensure comprehensive coverage of the search space.

[0083] The population consists of multiple agents, each composed of multiple individuals. During initialization, a random search is used to determine the positions of individuals in the search space; these positions are constrained, with upper and lower bounds defining the allowable range for each dimension. An initial population is randomly generated, with each individual representing a possible solution. The number of individuals in the population is determined based on the specific problem. The fitness values ​​of the individuals in the population are calculated, the objective function value for each individual is calculated, and the initial optimal solution is determined.

[0084] In the initial path generation phase, a Gaussian function is introduced to control the path distribution. By adjusting the parameters of the Gaussian function, paths with different characteristics can be generated in the solution space, thereby increasing the diversity of the initial population. Individual characteristics are represented by the Gaussian function, generating a series of variations around the potential optimal solution. The diffusion and individual characteristics in this strategy work together to allow the initial population to cover a wider range of the solution space while concentrating more closely in the region closer to the optimal solution, thus improving the algorithm's search efficiency and convergence speed.

[0085] The Gaussian function is expressed as: In the formula, The best individual in the current population. For the current solution, a1 and a2 are random distributions ranging from 0 to 1. Therefore Let μ1 be a numerical Gaussian distribution within a function with mean μ1 and standard deviation μ1; where μ1 = δ⊙(ub-lb); ⊙ represents the element-wise product of corresponding elements, the upper and lower bounds of the solution space are defined by vectors with values ​​ub and lb, respectively, and δ is a constant.

[0086] During the initial path generation phase, potential optimal solutions are determined based on the fitness values ​​of the current population; a Gaussian function is used to generate a set of variations around the potential optimal solutions; the generated set of variations is added to the population to update the individual positions of the population.

[0087] The frost factor and exploration factor are updated based on the current iteration number. When the frost factor is less than the preset adhesion coefficient, the agent's position is moved towards the current optimal solution. When the frost factor is greater than or equal to the preset adhesion coefficient, the agent's position is updated through random perturbation based on the relationship between the fitness value and the exploration factor. The adhesion coefficient represents the proximity between the best agent and the randomly selected agent, and is limited to the range [0,1].

[0088] In UAV path planning, updating the agent's position through random perturbations helps the UAV explore possible paths over a wider area, especially in complex terrain or dynamic environments. The algorithm dynamically adjusts the search direction and step size by simulating the process of water vapor condensing and falling at high altitudes. Introducing random perturbations to simulate this process in path planning allows the UAV to explore more distant areas, thereby increasing the likelihood of finding the globally optimal path.

[0089] Updating the agent's position through random perturbation includes the following steps: Updating the agent's position using the following formula: R(t) = R opt +(R avg -R prev )·L(Dim)·f(t); where, In the formula, R(t) represents the position of the agent in the t-th iteration, R opt For the best location of the agent, R avg R is the average value of the optimal location. prev Let L(Dim) be the previous best position of the agent, f(t) be the Lévy flight function, f(t) be the transition factor function, c be a random number between [0,1], ζ be a random number between [0,1], T be the total number of iterations, s and β be constants, σ ​​be the factor function, μ be a random number between [0,1], and v be a random number between [0,1].

[0090] Then, individuals in the population are randomly paired, and an arithmetic crossover operation is performed on the paired individuals to generate new individuals, and the positions of the new individuals are calculated. It should be noted that in the later stages of the algorithm, frozen individuals tend to cluster around the optimal solution, leading to a loss of population diversity. This may cause the algorithm to converge prematurely and fall into a local optimum. To avoid this special case, information exchange between paths is promoted by simulating the weaving and fusion process during crystal growth. In path planning, paths in the current population can be randomly paired, and new paths can be generated through an arithmetic crossover operation. This operation helps to retain the characteristics of excellent paths while avoiding falling into local optima. At the beginning of the algorithm, individual frost crystals within the population are randomly paired, and each pair is treated as two nodes in the lattice structure. The algorithm simulates the interconnection and information exchange process between nodes during grid growth and calculates the position of the two original individuals. i1 and RM i2 Perform arithmetic crossover. Two original individuals RM i1 and RM i2 Just like weaving new lattice connections, new individuals are created. and

[0091] Represented as:

[0092] Represented as:

[0093] RM i1j and RM i2j They represent RM respectively i1 RM i2 The j-th dimension (j = 1, 2, ..., D), and They represent RM respectively i1j The (j+1)th dimension and RM i2j The (j+1)th dimension of the second generation is generated by a lattice weaving strategy. α1 and α2 are random numbers randomly generated in the range (0,1). The values ​​of ω1 and ω2 are generated by a uniform distribution of random numbers between -1 and 1. The generated second generation is compared with the first generation to retain individuals with better positions.

[0094] The above strategies can preserve excellent frost crystal individuals, reduce unnecessary ineffective search times, and prevent the algorithm from converging too early and deviating from local optima.

[0095] Compare the updated fitness value of an individual with its original fitness value. Retain the individual with the better fitness value and replace the individual with the worse fitness value. For a pair of individuals, if one individual has a better fitness value, retain that individual, and replace the other individual based on the updated solution.

[0096] Determine if the maximum number of iterations has been reached or if other stopping conditions are met (such as the change in fitness value being less than a certain threshold). If the stopping conditions are met, output the current optimal solution and obtain the flight path; otherwise, return to the diffusion phase and continue iterating.

[0097] Images of poles and towers were acquired using drones, including multiple images of the same pole and tower taken at different times.

[0098] Preprocessing of tower images is crucial. Due to the processing methods used for low-altitude remote sensing images, target detection requires higher accuracy. Remote sensing images are inevitably affected by factors such as sensor pose, distance, and image quality. Therefore, before target detection, appropriate precise geometric correction, atmospheric correction, and cross-registration are necessary for remote sensing images from different periods. This improves the accuracy of target detection. Image registration aims to reduce errors caused by differences in shooting angles, prevent miscorrelation between the same spatial location in different images, and avoid comparing feature points extracted later by analyzing feature points from one image with feature points from other locations in another image, which could easily lead to a high false detection rate.

[0099] The preprocessed tower images are input into the detection network.

[0100] Reference Figure 5 The backbone of the detection network consists of four stages. Each stage extracts global and local features and then fuses the global and local features.

[0101] For step S300, the detection network constructs a dual-path feature extraction framework. Global features are extracted using a lightweight Transformer network, while local features are extracted using a reversible neural network layer. Feature difference is performed on global and local features by element-wise subtraction followed by an absolute value operation. A lightweight attention inversion module serves as the basic unit for feature extraction. By simplifying the feedforward network structure, the lightweight attention inversion module effectively reduces the embedding dimension and the number of parameters while maintaining comparable performance. Given the importance of detailed edge and texture features in image fusion tasks, the reversible neural network module enhances the preservation of input information by enabling bidirectional generation of input and output features.

[0102] Extracting global features from tower images includes: extracting global features using a lightweight Transformer network.

[0103] Extracting local features from tower images includes: extracting local features using a lightweight attention inversion module.

[0104] Reference Figure 6 The lightweight attention inversion module includes a local perception module, a lightweight multi-head attention module, and an inverted residual feedforward network; it enhances local feature extraction capabilities while reducing computational complexity and parameters. This contributes to the lightweighting of UAV inspection systems.

[0105] In the local perception module, the input of the local perception module is added to the result of deep convolution, and the result is normalized by the layer to obtain the output of the local perception module. The output of the local perception module is used as the input of the lightweight multi-head attention module.

[0106] In the lightweight multi-head attention module, the input of the lightweight multi-head attention module is processed by a first linear mapping, and the result of the first linear mapping is used as the query of the multi-head attention mechanism. The input of the lightweight multi-head attention module is processed by a kxk depth convolution with stride k and a second linear mapping, and the result of the second linear mapping is used as the key of the multi-head attention mechanism. The input of the lightweight multi-head attention module is processed by a kxk depth convolution with stride k and a third linear mapping, and the result of the third linear mapping is used as the value of the multi-head attention mechanism. The multi-head attention mechanism obtains attention features based on the query, key, and value. The attention features are normalized by layers to obtain the output of the lightweight multi-head attention module. The output of the lightweight multi-head attention module is used as the input of the inverted residual feedforward network.

[0107] In the inverted residual feedforward network, the input of the inverted residual feedforward network is convolved through multiple convolutional layers to obtain the output of the inverted residual feedforward network.

[0108] To maintain translation invariance in data augmentation while focusing on local relevance and structural information, a local perception module is inserted before the multi-head attention mechanism. This module extracts local information from the input image through depthwise convolution and residual connections, preserving translation invariance in data augmentation while focusing on local relevance and structural information. Depthwise convolution reduces the number of parameters and computational cost compared to traditional convolution, contributing to the goal of lightweight algorithm design. This lightweight design is particularly important for UAV inspection systems, as UAVs typically carry embedded devices with limited computing power and storage resources. By reducing the number of model parameters and computational complexity, the lightweight algorithm can significantly improve the UAV's endurance and real-time processing capabilities, enabling it to operate efficiently in complex environments, thereby enhancing inspection efficiency and reliability.

[0109] In the multi-head self-attention module, to save computational costs, a depthwise convolution with a stride of k is used to reduce the size of the key and value. Furthermore, the dimensions of the query and key are halved. Specifically, in this module, the input X∈R is first mapped using a linear mapping. n×d Project to query key Sum Then, depthwise convolution is used to reduce the dimensionality of the keys and values, resulting in lightweight keys. and lightweight values In addition, the LWT block uses a relative position bias B, which can be learned in each self-attention module.

[0110] The lightweight multi-head self-attention module is implemented as follows:

[0111] Among them, bias By combining random initialization with relative positional biases, multi-head self-attention can better learn local correlations in an image. Ultimately, each head outputs a value of size [value missing]. The sequences, where h is the number of heads, are concatenated into a final sequence of size n×d.

[0112] To better perform feature extraction and improve performance, the position of the feedforward network in ViT is changed in the inverted residual feedforward network. The feedforward network contains an activation layer followed by batch normalization. The intermediate convolutions are depthwise convolutions, which capture region features at a lower cost. In addition, a shortcut connection is used between the first and last 1×1 convolutions to improve propagation capability.

[0113] The formula for a feedforward network is as follows:

[0114] Here, Act(·) represents the activation layer, followed by batch normalization. The convolutions in the middle are depthwise convolutions, used to capture region features. In addition, shortcut connections are used to improve propagation capability.

[0115] Through the synergistic effect of the local perception module, the lightweight multi-head attention module, and the inverted residual feedforward network, the lightweight attention inversion module effectively extracts and fuses local and global features while maintaining high computational efficiency, significantly improving the model's performance in detection tasks and successfully achieving algorithm lightweighting. This lightweight design is crucial for UAV inspection systems because it not only reduces computational resource consumption but also lowers energy consumption, thereby extending the UAV's endurance and enhancing its real-time processing capabilities. This means that UAVs can cover larger inspection areas for longer periods, while reducing reliance on high-end hardware, lowering costs, and improving system reliability and deployability. Furthermore, the high computational efficiency helps to quickly detect minute deformations of poles and towers.

[0116] Reference Figure 3 For step S400, key features are identified from global and local features through frequency channel attention and frequency spatial attention, including the following steps:

[0117] Step S410: Aggregate the features along the channel dimension using global pooling to obtain the first aggregated feature;

[0118] Step S420: Separate the response components of different channels to key regions in the aggregated features;

[0119] Step S430: Extract a first low-frequency feature and a first high-frequency feature based on the response components, wherein the frequency of the first low-frequency feature is less than the frequency of the first high-frequency feature;

[0120] Step S440: Convolve the first low-frequency feature and the first high-frequency feature to obtain the channel weights of each channel;

[0121] Step S450: Aggregate the features in the spatial dimension using global pooling to obtain the second aggregated feature;

[0122] Step S460: Perform convolution and residual connection on the second aggregated feature to obtain position information;

[0123] Step S470: Perform fast Fourier transform and inverse Fourier transform on the second aggregated feature to obtain the second low-frequency feature and the second high-frequency feature. The frequency of the second low-frequency feature is less than the frequency of the second high-frequency feature.

[0124] Step S480: Convolve the second low-frequency feature and the second high-frequency feature to obtain the spatial weights of each space;

[0125] Step S490: Key features are identified based on channel weights and spatial weights.

[0126] The channel weights of each channel are obtained by executing steps S410, S420, S430, and S440 by the frequency channel module; the spatial weights of each space are obtained by executing steps S450, S460, S470, and S480 by the frequency space module.

[0127] In the frequency channel module, channel information is aggregated through global pooling along the channel dimension. The aggregation process is represented as follows: Where y represents the aggregated channel features, Cat(·) is the operation of concatenating tensors along the specified dimension, q1 and q2 represent the biphase features after processing by the frequency channel module, AVg(·) and Max(·) correspond to the global average pooling and global max pooling operations in the channel dimension, Ca(·) is the frequency domain channel module, T(·) represents the Transformer encoder, I(·) represents the invertible neural network (INN), t represents the global feature, and i represents the local feature.

[0128] Traditional attention mechanisms often struggle to fully preserve the frequency domain components of multi-source features. To address this deficiency and maintain complementary feature information, a frequency channel module is employed from a frequency domain analysis perspective to separate the response components of different channels to key regions. Its mathematical representation is: Ca(t)=σ(FC(ReLU(FC(DCT(t)))))); where σ(·) represents the sigmoid activation function, FC(·) represents the fully connected layer operation, and DCT(·) corresponds to the two-dimensional discrete cosine transform. The discrete cosine transform (DCT) converts image data from the time domain to the frequency domain and expresses signal features through a linear combination of the data.

[0129] Two-dimensional discrete cosine transform (DCT) is used for low-frequency feature extraction because it concentrates the main energy in the low-frequency components during the signal frequency domain transformation, while high-frequency components typically exhibit smaller amplitude characteristics. The two-dimensional DCT process is expressed as follows: Where h∈{0,1,…,H-1}, w∈{0,1,…,W-1}.

[0130] Channel features are processed through two one-dimensional convolutional layers to calculate the weight distribution of each channel within the feature. These weights are then normalized using a softmax function, ensuring their sum equals 1, thus enabling comparability analysis of the weights for the two-temporal features. By comparing the weight values, channels with higher significance can be identified, thereby highlighting the most critical information components in the two-temporal channel features. The generated channel weights can be expressed as: w c1 ,w c2 =σ(ECA(y)); where σ(·) represents the sigmoid activation function, w c1 and w c2These represent the channel weights in the two-phase time, and ECA(·) represents a one-dimensional convolution operation.

[0131] Specifically, the input features are processed by 2D discrete cosine transform, fully connected layer, ReLU activation function, fully connected layer and sigmoid activation function, and the input features are skip-connected to obtain the output features by multiplying the input features and the processing results.

[0132] In the frequency space module, global features and spatial information are integrated through a global pooling operation along the spatial dimension, which can be expressed as: Where z represents the spatial feature of the aggregation, and Sa(·) is the frequency space module.

[0133] To enhance the fusion of spatial and frequency information, a frequency spatial module, denoted as Sa(·), is designed. This module is specifically designed to extract and integrate features from the spatial and frequency domains for a more comprehensive understanding of the data. The introduction of this module not only optimizes feature aggregation but also reduces the spatial and temporal complexity of the processing.

[0134] The frequency space module can be represented as: Where σ(·) is the sigmoid activation function, Conv 1×1 (·) represents a 1×1 convolution operation, and W is a learnable weighted gating mechanism whose parameter settings are related to the width and height dimensions of the input image.

[0135] The frequency space module consists of two core components: a location module, which refines features through residual connections and convolutions to accurately capture location details; and a frequency domain module, composed of Fast Fourier Transform (FFT) and Inverse Fourier Transform (IFT) layers, which extract different information by introducing frequency components to form a multispectral description and fully utilize the information. The FFT layer extracts high-frequency features by decomposing the signal into different frequency components and introducing a learnable weighted gating W to select useful high- and low-frequency components. Specifically, in one branch, the input features are subjected to an FFT, and useful high- and low-frequency components are selected through weighted gating, followed by an IFT. In another branch, the input features are subjected to a 1x1 convolution and a sigmoid activation function operation. The input features and the operation results are then skip-connected to multiply the two results. The results from both branches are then summed to obtain the output features.

[0136] The generated spatial weights can be represented as: w s1 ,w s2 =σ(Conv(z)); where σ(·) represents the sigmoid function, w s1 and w s2Represents the spatial weights of the two temporal phases, and Conv(·) represents a 1x1 convolution.

[0137] At each stage, dual-temporal images (two images of the same tower taken at different times) are input into the LWT module. The LWT module performs feature difference on global and local features by subtracting the two results from the dual-temporal images element-wise and then taking the absolute value. Similarly, the two results from the dual-temporal images are input into the reversible neural network, and then perform feature difference on global and local features by subtracting the absolute value. The two difference results are input into the frequency channel module and the frequency space module, respectively, to calculate the channel weights and spatial weights. The results from the frequency channel module and the frequency space module are then subjected to average pooling and max pooling. The pooling results in the channel dimension are convolved by the convolution module, and the pooling results in the spatial dimension are used to extract attention features through the efficient channel attention module. The convolution results and attention features are then fused, and the fused features are input into the progressively modified detection module.

[0138] The channel and spatial weights of the bi-temporal features are fused to create a comprehensive set of bi-temporal weights, thereby highlighting the key components of the bi-temporal features. These weights are then used to multiply and combine the bi-temporal features to achieve seamless fusion of temporal information. The final output is represented as: r = (w c1 +w s1 )*t+(w c2 +w s2 )*i; where r represents the fused feature. This process prioritizes key elements in the dual-temporal features while filtering out less important parts, thereby achieving more refined and effective feature fusion.

[0139] For step S500, multi-scale features are obtained by progressively modifying the detection module to perform dilation convolution operations on key features with different dilation rates.

[0140] Reference Figure 4 The dilated convolution operation includes the following steps:

[0141] Step S510: The input of the dilated convolutional layer is subjected to average pooling, grouped convolution, and classification to obtain classification features;

[0142] Step S520: Perform grouped convolution on the input of the dilated convolutional layer to obtain grouped convolutional features;

[0143] Step S530: The result of multiplying the classification features and grouped convolution features is reshaped and convolved to obtain the output of the dilated convolutional layer.

[0144] The performance of convolutional neural networks is limited by traditional convolutional operations, which rely on shared parameters and are not sensitive enough to information differences caused by positional variations. To address this, dilated convolution operations with varying dilation rates on key features can capture feature information at different scales, which is crucial for target detection and subtle deformation recognition in complex environments by UAVs. Furthermore, dilated group convolutions reduce the number of parameters and computational complexity through grouped convolutions, while enhancing the model's ability to extract local features. This characteristic makes dilated group convolutions perform exceptionally well in lightweight models mounted on UAVs.

[0145] Reference Figure 7 In the progressively modified detection module, the input feature r is subjected to a dilated convolution with a dilation rate of 5 (d=5). The dilated convolution result is multiplied element-wise with the input feature. The multiplication result is activated by the ReLU activation function. The activation result is subjected to a dilated convolution with a dilation rate of 3 (d=3). The dilated convolution result is added element-wise with the input feature. The addition result is activated by the ReLU activation function. The activation result is subjected to a dilated convolution with a dilation rate of 1 (d=1). The dilated convolution result is added element-wise with the input feature. The addition result is activated by the ReLU activation function to obtain the output feature u.

[0146] Dilated group convolutions address these issues by prioritizing the importance of different features within the receptive field and emphasizing the spatial properties of that field. Therefore, the feature maps generated by dilated group convolutions are better at capturing spatial details because the learned attention mechanism is tailored to each specific receptive field, avoiding the previous problem of sharing feature information across fields. Due to the use of group convolutions, the height and width of the shaped features are increased threefold, and 3×3 dilated convolutions with different dilation rates are used to further extract feature information.

[0147] The formula for dilated group convolution is expressed as: F d (r)=Conv d (Softmax(Convg 1×1 (Avg(r)))*ReLU(Norm(Convg 3×3 (r))));where d=1,3,5,Convg 1×1 (·) represents a 1×1 grouped convolution, Convg 3×3 (·) denotes a 3×3 grouped convolution, Norm(·) denotes a normalization operation, x is the input feature map, and Conv d (·) represents a dilated convolution with an expansion rate of d.

[0148] The Detection Module (SMDM) is progressively modified to better capture time-varying information and improve the integrity of features within changing regions. This module can simultaneously consider information from multiple receptive fields and performs multi-scale feature learning through three dilated group convolutional modules with different dilation rates connected to residuals.

[0149] SMDM is defined by the following formula: r' = ReLU(r + F d (r)),d=5;r”=ReLU(r+F d (r')),d=3;u=ReLU(r+F d (r”)),d=1; where r represents the input of the SMDM and u represents the output of the SMDM.

[0150] By employing a trapezoidal structure, SMDM systematically explored the temporal variations of different receptive field sizes, from large to small. This ensured that the interconnect branches mutually reinforced each other in detecting these variations.

[0151] For step S600, the supervised learning module performs supervised learning based on multi-scale features to integrate semantic and geometric information and generate supervised attention features.

[0152] In traditional layer-by-layer feature extraction, high-level networks excel at capturing semantic information, but the generated feature maps have low resolution, limiting their ability to represent geometric details. Conversely, low-level networks, due to their high-resolution feature maps, are adept at capturing geometric details but perform poorly in encoding semantic information. Supervised learning modules refine and enhance features, enabling more effective multi-layer integration. These modules effectively integrate high-level semantic information with low-level geometric details, aiding UAVs in target detection and subtle deformation recognition in complex environments. Furthermore, through feature refinement and enhancement, the supervised learning module improves the model's focus on key features, reduces noise from multi-layer aggregation, and eliminates irrelevant information. This makes the supervised learning module perform exceptionally well in lightweight models mounted on UAVs, particularly when processing high-resolution images, highlighting key features and filtering out unimportant information.

[0153] Reference Figure 8 In one branch of the supervised learning module, the input feature u passes through a fully connected layer and a ReLU activation function; in the other branch of the supervised learning module, the input feature passes through a fully connected layer, the output of the fully connected layer passes through a 1x1 convolution, the convolution result is inverted, the convolution result and the inverted result are concatenated, the concatenation result passes through a Sigmoid activation function, the results of the two branches are multiplied element-wise, and the multiplication result is concatenated through a 3x3 convolution to obtain the output feature m.

[0154] Specifically, fully connected layers and ReLU activation layers are added to condense spatial information into a dense representation. This improves the model's focus on key features and reduces noise from multi-layer aggregation, eliminating irrelevant information. Fully connected layers and convolutions are applied, followed by feature generation via a sigmoid activation function. Then, an inverse transformation map is computed through reverse processing. This allows for the representation of changing regions while capturing stable regions. Finally, the transformations are concatenated and an attention mask d is generated through a convolutional layer.

[0155] The supervised learning module can be represented by the following formula: Where u is the input feature, σ(·) represents the sigmoid function, and Cat(·) is an operation that connects two tensors along a specified dimension.

[0156] Attention-weighted features are generated using 'a', thereby refining the features. This process is represented as: m = Conv 33 (ReLU(FC(u))*a); 'a' acts as a filtering mechanism on the input u, retaining only features relevant to the change. Finally, the supervised learning module generates supervised attention to guide the further fusion of features m with higher-level features.

[0157] For step S700, the supervised attention features are decoded by the decoder to generate a change segmentation map, and the target detection is performed based on the change segmentation map to obtain the tower deformation detection result.

[0158] The decoder consists of four LWT layers, which correspond to the layers in the encoder and use the same number and embedding dimensions in reverse order. Each layer of the decoder contains the same number of LWT blocks as the encoder and has an upsampling module that progressively restores the original size of the image through convolutional operations and integrates features at different scales through skip connections.

[0159] The decoder takes a multi-scale feature map as input, processed by the fusion module. In each layer of the decoder, the input feature map is first processed by an LWT block, then fed into a patch merging module. This module involves linear upsampling to double the size and convolution to halve the number of channels. It is then added to a fused feature map of the same size and dimensions. In this way, the decoder extracts multi-scale variation information to generate the final variation segmentation map.

[0160] The two sets of images captured by the drone are preprocessed, then input into a trained target detection network, and then output a segmentation map of the changes. This map is then transmitted to the relevant application of the staff via the Internet, and finally the staff further determines whether there are any changes in the poles in the corresponding area after the disaster.

[0161] During the training of the detection network, a loss function is obtained based on the tower deformation detection results; the parameters of the detection network are adjusted according to the loss function; the loss function of the detection network is the sum of the classification loss and the weighted cross-entropy loss.

[0162] A hybrid loss function is defined, which combines classification loss and weighted cross-entropy loss to optimize model performance. The loss function is defined as: L = λ·L wce +(1-λ)·L dice Where λ is the weighting parameter, L wce and L dice denoted as weighted cross-entropy loss and classification Dice loss, respectively.

[0163] L wce Represented as:

[0164] L dice Represented as:

[0165] L wce Used to solve the class imbalance problem, L dice Used to evaluate the similarity between the predicted map and the real map.

[0166] By adjusting the parameters of the detection network using a loss function, the accuracy of the detection network in detecting tower deformation can be improved.

[0167] An embodiment of this application provides an electronic device. The electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-described UAV-based pole deformation detection method.

[0168] This electronic device can be any smart terminal, including computers.

[0169] In general, for the hardware structure of electronic devices, the processor can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, to execute relevant programs and implement the technical solutions provided in the embodiments of this application.

[0170] The memory can be implemented in the form of read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory and is called and executed by the processor.

[0171] Input / output interfaces are used to implement information input and output.

[0172] The communication interface is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0173] The bus transmits information between various components of a device, such as the processor, memory, input / output interfaces, and communication interfaces. The processor, memory, input / output interfaces, and communication interfaces communicate with each other within the device via the bus.

[0174] An embodiment of this application provides a computer storage medium. The computer storage medium stores computer-executable instructions for executing the UAV-based pole deformation detection method described above.

[0175] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium. In the foregoing description of this specification, references to terms such as "one embodiment," "another embodiment," or "some embodiments," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of this application. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0176] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0177] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0178] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0179] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0180] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed between each other may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms. Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

[0181] The above is a detailed description of the preferred embodiments of this application, but this application is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application, and these equivalent modifications or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A method for detecting pole deformation based on unmanned aerial vehicles (UAVs), characterized in that, include: Images of poles and towers are acquired by drones and input into the detection network. The images of poles and towers include multiple images of the same pole and tower at different times. Extract the global and local features of the tower image; Key features are identified from the global and local features using frequency channel attention and frequency spatial attention. Multi-scale features are obtained by performing dilation convolution operations with different dilation rates on the key features; Supervised learning is performed based on the multi-scale features to integrate semantic and geometric information and generate supervised attention features. The supervised attention features are decoded to generate a change segmentation map, and the target detection is performed based on the change segmentation map to obtain the tower deformation detection result; The step of identifying key features from the global and local features through frequency channel attention and frequency spatial attention includes: The first aggregated feature is obtained by aggregating through global pooling along the channel dimension. Separate the response components of different channels to key regions in the first aggregated feature; A first low-frequency feature and a first high-frequency feature are extracted based on the response components, wherein the frequency of the first low-frequency feature is less than the frequency of the first high-frequency feature; Convolve the first low-frequency feature and the first high-frequency feature to obtain the channel weights for each channel; The second aggregated feature is obtained by aggregating the features in the spatial dimension through global pooling. The second aggregated feature is convolved and residual connected to obtain the location information; The second aggregated feature is subjected to fast Fourier transform and inverse Fourier transform to obtain the second low-frequency feature and the second high-frequency feature, wherein the frequency of the second low-frequency feature is less than the frequency of the second high-frequency feature. Convolve the second low-frequency feature and the second high-frequency feature to obtain the spatial weights of each space; Key features are identified based on the channel weights and spatial weights.

2. The method for detecting pole deformation based on unmanned aerial vehicles according to claim 1, characterized in that, Plan the flight path of the drone according to the following steps: Initialize a population, which includes multiple agents, each agent consisting of multiple individuals; Calculate the fitness value of individuals in the population; Update the frost layer factor and exploration factor based on the current iteration number; When the frost factor is less than the preset adhesion coefficient, the agent's position is moved toward the current optimal solution; When the frost factor is greater than or equal to the preset adhesion coefficient, the agent's position is updated by random perturbation according to the relationship between the fitness value and the exploration factor. Randomly pair individuals in the population, perform arithmetic crossover on the paired individuals to generate new individuals, and calculate the position of the new individuals; Replace individuals based on their fitness values ​​before and after the update; Output the optimal path based on the location of the replaced individual.

3. The method for detecting pole deformation based on unmanned aerial vehicles according to claim 1, characterized in that, The method of updating the agent's location through random perturbation includes: Update the agent's position using the following formula: ;in, , , ; In the formula, Let be the position of the agent in the t-th iteration. For the best location of the agent, The average value of the optimal position. The best position for the agent. For the Lévy flight function, This represents the transition factor function, where c is a random number between [0,1]. Let be a random number between [0,1], T be the total number of iterations, and s and It is a constant. For factor functions, A random number between [0,1] It is a random number between [0,1].

4. The method for detecting pole deformation based on unmanned aerial vehicles according to claim 1, characterized in that, Extracting global features from the tower image includes: extracting global features using a lightweight Transformer network.

5. The method for detecting tower deformation based on unmanned aerial vehicles according to claim 1, characterized in that, Extracting local features from the tower image includes: extracting local features using a lightweight attention inversion module; The lightweight attention inversion module includes a local perception module, a lightweight multi-head attention module, and an inverted residual feedforward network. In the local perception module, the input of the local perception module is added to the result of deep convolution, and the result is normalized by the layer to obtain the output of the local perception module. The output of the local perception module is used as the input of the lightweight multi-head attention module. In the lightweight multi-head attention module, the first linear mapping result of the input of the lightweight multi-head attention module after linear mapping is used as the query of the multi-head attention mechanism; the second linear mapping result of the input of the lightweight multi-head attention module after depthwise convolution and linear mapping is used as the key of the multi-head attention mechanism; the third linear mapping result of the input of the lightweight multi-head attention module after depthwise convolution and linear mapping is used as the value of the multi-head attention mechanism; attention features are obtained through the multi-head attention mechanism based on the query, key, and value; the attention features are normalized by layers to obtain the output of the lightweight multi-head attention module; and the output of the lightweight multi-head attention module is used as the input of the inverted residual feedforward network. In the inverted residual feedforward network, the input of the inverted residual feedforward network is convolved through multiple convolutional layers to obtain the output of the inverted residual feedforward network.

6. The method for detecting pole deformation based on unmanned aerial vehicles according to claim 1, characterized in that, The dilated convolution operation includes: The input to the dilated convolutional layer is subjected to average pooling, grouped convolution, and classification to obtain classification features; Grouped convolutional features are obtained by performing grouped convolution on the input of the dilated convolutional layer. The result of multiplying the classification features and the grouped convolutional features is reshaped and convolved to obtain the output of the dilated convolutional layer.

7. The method for detecting pole deformation based on unmanned aerial vehicles according to claim 1, characterized in that, The method further includes: The loss function is obtained based on the tower deformation detection results; The parameters of the detection network are adjusted according to the loss function; The loss function of the detection network is the sum of the classification loss and the weighted cross-entropy loss.

8. An electronic device, characterized in that, include: The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the UAV-based pole deformation detection method as described in any one of claims 1 to 7.

9. A computer storage medium, characterized in that, The device stores computer-executable instructions for performing the UAV-based pole deformation detection method as described in any one of claims 1 to 7.

Images