Pole tower deformation detection method and equipment based on unmanned aerial vehicle, and medium

Abstract

The invention provides a tower deformation detection method and device based on an unmanned aerial vehicle and a medium. The method comprises the steps of obtaining a tower image through an unmanned aerial vehicle, and extracting global features and local features of the tower image; identifying key features of the global features and the local features through frequency channel attention and frequency space attention; performing expansion convolution operation with different expansion rates on the key features to obtain multi-scale features; performing supervised learning according to the multi-scale features to integrate the semantic information and the geometric information, and generating supervised attention features; decoding the supervision attention features to generate a change segmentation map, and performing target detection according to the change segmentation map to obtain a tower deformation detection result; efficient, accurate and automatic tower deformation detection is realized, climbing detection by a detector is not needed, and the safety is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing, and in particular to a method, device, and medium for detecting tower deformation based on a drone. Background Art

[0002] Pole towers are supporting structures in power or communications systems, typically constructed of steel or concrete, used to support high-voltage transmission lines, communication cables, or signaling equipment. Their primary function is to support conductors, maintain safe distances, and ensure stable transmission of power or signals. Natural disasters can easily damage towers, causing them to deform. Even slight deformation can disrupt their load balance, reduce structural stability, and lead to fatigue damage to tower materials, loosening of connections, and even collapse. Furthermore, slight deformation can alter the stress distribution of conductors, affecting safe distances for electrical equipment and increasing the risk of conductor breakage, equipment failure, and line tripping, threatening the continuity and reliability of power transmission. Therefore, post-disaster inspection and assessment of tower deformation is essential. Traditional manual inspections require inspectors to climb and perform inspections, which poses safety risks, is time-consuming, and incurs high labor costs. Summary of the Invention

[0003] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0004] The purpose of this application is to solve one of the technical problems existing in the related art to at least a certain extent. The embodiments of this application provide a pole tower deformation detection method, equipment and medium based on drones, which realizes efficient, accurate and automated detection.

[0005] An embodiment of the first aspect of the present application is a method for detecting tower deformation based on a drone, comprising:

[0006] Acquire tower images through a drone, and input the tower images into the detection network, wherein the tower images include multiple images of the same tower at different times;

[0007] Extracting global features and local features of the tower image;

[0008] Identifying key features of the global features and the local features through frequency channel attention and frequency space attention;

[0009] Performing dilated convolution operations with different dilation rates on the key features to obtain multi-scale features;

[0010] Performing supervised learning based on the multi-scale features to integrate semantic information and geometric information to generate supervised attention features;

[0011] The supervised attention feature is decoded to generate a change segmentation map, and target detection is performed based on the change segmentation map to obtain a tower deformation detection result.

[0012] According to certain embodiments of the first aspect of the present application, the flight path of the drone is planned according to the following steps:

[0013] Initializing a population, wherein the population includes a plurality of agents, and the agents are composed of a plurality of individuals;

[0014] Calculate the fitness value of individuals in the population;

[0015] Update the frost factor and exploration factor based on the current number of iterations;

[0016] When the frost factor is less than a preset adhesion coefficient, the agent's position is moved toward the current optimal solution;

[0017] When the frost factor is greater than or equal to a preset adhesion coefficient, the position of the agent is updated by random perturbation according to the numerical relationship between the fitness value and the exploration factor;

[0018] Randomly pair individuals in the population, perform arithmetic crossover operations on the paired individuals to generate new individuals, and calculate the positions of the new individuals;

[0019] Replace individuals according to their fitness values ​​before and after update;

[0020] Output the optimal path based on the position of the replaced individual.

[0021] According to certain embodiments of the first aspect of the present application, updating the position of the agent by random perturbation includes:

[0022] Update the agent's position by: R(t) = R opt +(R avg -R prev )·L(Dim)·f(t); where,

[0023] Where R(t) is the position of the agent in the tth iteration, R opt is the optimal position of the agent, R avg is the average value of the best position, R prev is the agent's previous best position, L(Dim) is the Levy flight function, f(t) represents the transition factor function, c is a random number between [0,1], ζ is a random number between [0,1], T is the total number of iterations, s and β are constants, σ ​​is the factor function, μ is a random number between [0,1], and v is a random number between [0,1].

[0024] According to certain embodiments of the first aspect of the present application, extracting global features of the tower image includes: extracting the global features through a lightweight Transformer network.

[0025] According to certain embodiments of the first aspect of the present application, extracting local features of the tower image includes: extracting local features using a lightweight attention inversion module;

[0026] The lightweight attention inversion module includes a local perception module, a lightweight multi-head attention module and an inverted residual feedforward network;

[0027] In the local perception module, the input of the local perception module is added to the result of deep convolution of the input of the local perception module, and the addition result is layer normalized to obtain the output of the local perception module. The output of the local perception module serves as the input of the lightweight multi-head attention module;

[0028] In the lightweight multi-head attention module, the input of the lightweight multi-head attention module is subjected to a first linear mapping result of linear mapping as a query of the multi-head attention mechanism, the input of the lightweight multi-head attention module is subjected to a second linear mapping result of depth convolution and linear mapping as a key of the multi-head attention mechanism, the input of the lightweight multi-head attention module is subjected to a third linear mapping result of depth convolution and linear mapping as a value of the multi-head attention mechanism, and the attention feature is obtained according to the query, key and value through the multi-head attention mechanism, the attention feature is normalized by the layer to obtain the output of the lightweight multi-head attention module, and the output of the lightweight multi-head attention module is used as the input of the inverted residual feedforward network;

[0029] In the inverted residual feedforward network, the input of the inverted residual feedforward network is convolved through multiple convolutional layers to obtain the output of the inverted residual feedforward network.

[0030] According to certain embodiments of the first aspect of the present application, identifying key features of the global features and the local features through frequency channel attention and frequency space attention includes:

[0031] Aggregate through global pooling in the channel dimension to obtain the first aggregated feature;

[0032] separating response components of different channels to the key regions in the aggregated features;

[0033] extracting a first low-frequency feature and a first high-frequency feature according to the response component, wherein the frequency of the first low-frequency feature is lower than the frequency of the first high-frequency feature;

[0034] Convolving the first low-frequency feature and the first high-frequency feature to obtain a channel weight of each channel;

[0035] Aggregate through global pooling in the spatial dimension to obtain the second aggregated feature;

[0036] Performing convolution and residual connection on the second aggregated features to obtain position information;

[0037] Performing a fast Fourier transform and an inverse Fourier transform on the second aggregated feature to obtain a second low-frequency feature and a second high-frequency feature, wherein the frequency of the second low-frequency feature is less than the frequency of the second high-frequency feature;

[0038] Convolving the second low-frequency feature and the second high-frequency feature to obtain a spatial weight of each space;

[0039] Key features are identified based on the channel weights and the spatial weights.

[0040] According to certain embodiments of the first aspect of the present application, the dilated convolution operation includes:

[0041] The input of the dilated convolutional layer is average pooled, grouped, and classified to obtain classification features;

[0042] Perform group convolution on the input of the dilated convolution layer to obtain group convolution features;

[0043] The result of multiplying the classification feature and the grouped convolution feature is reshaped and convolved to obtain the output of the dilated convolution layer.

[0044] According to certain embodiments of the first aspect of the present application, the method further comprises:

[0045] Obtaining a loss function according to the tower deformation detection result;

[0046] Adjusting parameters of the detection network according to the loss function;

[0047] The loss function of the detection network is the sum of the classification loss and the weighted cross entropy loss.

[0048] An embodiment of the second aspect of the present application is an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the drone-based tower deformation detection method as described in the embodiment of the first aspect of the present application is implemented.

[0049] An embodiment of the second aspect of the present application is a computer storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the drone-based tower deformation detection method as described in the embodiment of the first aspect of the present application.

[0050] The above scheme has at least the following beneficial effects: acquiring tower images through drones and extracting global and local features of the tower images; identifying key features of global and local features through frequency channel attention and frequency space attention; performing dilated convolution operations with different dilation rates on key features to obtain multi-scale features; performing supervised learning based on multi-scale features to integrate semantic information and geometric information to generate supervised attention features; decoding the supervised attention features to generate a change segmentation map, performing target detection based on the change segmentation map to obtain tower deformation detection results; achieving efficient, accurate and automated tower deformation detection, without the need for inspectors to climb for inspection, thereby improving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The accompanying drawings are used to provide a further understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0052] Figure 1 It is a step diagram of the tower deformation detection method;

[0053] Figure 2 It is a diagram of the steps for planning the flight path of the drone;

[0054] Figure 3 is a sub-step diagram of step S400;

[0055] Figure 4 is the sub-step graph of the dilated convolution operation;

[0056] Figure 5 It is the structural diagram of the detection network;

[0057] Figure 6 This is the structural diagram of the lightweight attention inversion module;

[0058] Figure 7 It is a structural diagram of the gradual modification of the detection module;

[0059] Figure 8 It is the structural diagram of the supervised learning module. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0061] It should be noted that although the device schematics illustrate functional module divisions and the flowcharts illustrate logical sequences, in certain circumstances, the steps shown or described may be performed in a sequence that differs from the module divisions in the device or the sequence in the flowcharts. The terms "first," "second," and the like in the specification, claims, or accompanying drawings are used to distinguish similar items and are not necessarily used to describe a specific sequence or precedence.

[0062] The embodiments of the present application are further described below with reference to the accompanying drawings.

[0063] The embodiments of the present application provide a method for detecting tower deformation based on a drone.

[0064] Reference Figure 1 , a tower deformation detection method includes the following steps:

[0065] Step S200: acquiring a tower image through a drone and inputting the tower image into a detection network;

[0066] Step S300, extracting global features and local features of the tower image;

[0067] Step S400, identifying key features of global features and local features through frequency channel attention and frequency space attention;

[0068] Step S500, performing dilated convolution operations with different dilation rates on the key features to obtain multi-scale features;

[0069] Step S600 , performing supervised learning based on the multi-scale features to integrate semantic information and geometric information to generate supervised attention features;

[0070] Step S700: Decode the supervised attention features to generate a change segmentation map, and perform target detection based on the change segmentation map to obtain a tower deformation detection result.

[0071] It realizes efficient, accurate and automated detection of tower deformation, eliminating the need for inspectors to climb for inspection, thus improving safety.

[0072] The drone flies along the planned flight path and takes images of the tower along the way.

[0073] Reference Figure 2 , follow these steps to plan the flight path of your drone:

[0074] Step S110, initializing the population;

[0075] Step S120, calculating the fitness values ​​of individuals in the population;

[0076] Step S130, updating the frost layer factor and the exploration factor based on the current number of iterations;

[0077] Step S140 , when the frost factor is less than the preset adhesion coefficient, the agent's position is moved toward the current optimal solution;

[0078] Step S150 , when the frost factor is greater than or equal to the preset adhesion coefficient, the agent's position is updated by random perturbation according to the numerical relationship between the fitness value and the exploration factor;

[0079] Step S160, randomly pairing individuals in the population, performing an arithmetic crossover operation on the paired individuals to generate new individuals, and calculating the positions of the new individuals;

[0080] Step S170, replacing the individual according to the fitness value before and after the individual update;

[0081] Step S180: outputting the optimal path according to the position of the replaced individual.

[0082] In drone path planning, the initial population represents the possible flight paths of the drone. Diverse paths can be generated in the initial stages of path planning to ensure comprehensive coverage of the search space.

[0083] The population consists of multiple agents, each composed of multiple individuals. During initialization, random search is used to determine the positions of individuals in the search space; these positions are constrained, with upper and lower bounds defining the permissible range for each dimension. The initial population is randomly generated, with each individual representing a possible solution. The number of individuals in the population is set based on the specific problem. The fitness values ​​of the individuals in the population are calculated, along with the objective function value for each individual, to determine the initial optimal solution.

[0084] During the diffusion phase of generating initial paths, a Gaussian function is introduced to control the distribution of paths. By adjusting the parameters of the Gaussian function, paths with different characteristics can be generated within the solution space, thereby increasing the diversity of the initial population. Individual characteristics are represented by Gaussian functions, generating a series of variations around the potential optimal solution. The diffusion and individual characteristics in this strategy work together to enable the initial population to cover a wider range of the solution space while concentrating more closely on areas near the optimal solution, thereby improving the algorithm's search efficiency and convergence speed.

[0085] The Gaussian function is expressed as: Where, is the optimal individual in the current population, is the current solution, a1 and a2 are random distributions ranging from 0 to 1, Therefore is a digital Gaussian distribution within a function of mean and standard deviation μ1; where μ1 = δ⊙(ub-lb); ⊙ represents the element-by-element product of the corresponding elements, the upper and lower limits of the solution space are defined by vectors with values ​​ub and lb, respectively, and δ is a constant.

[0086] In the stage of diffusion generating the initial path, the potential optimal solution is determined based on the fitness value of the current population; a Gaussian function is used to generate a variation set around the potential optimal solution; the generated variation set is added to the population and the individual positions of the population are updated.

[0087] The frost factor and exploration factor are updated based on the current number of iterations. When the frost factor is less than the preset adhesion coefficient, the agent's position is moved toward the current optimal solution. When the frost factor is greater than or equal to the preset adhesion coefficient, the agent's position is updated through random perturbations based on the numerical relationship between the fitness value and the exploration factor. The adhesion coefficient represents the degree of proximity between the optimal agent and the randomly selected agent and is limited to the range [0,1].

[0088] In drone path planning, a strategy that updates the agent's position through random perturbations can help drones explore a wider range of possible paths, especially in complex terrain or dynamic environments. The algorithm simulates the process of water vapor condensing and falling at high altitudes, dynamically adjusting the path search direction and step size. By introducing random perturbations to simulate this process during path planning, drones can explore further, increasing the likelihood of finding the globally optimal path.

[0089] Updating the agent's position by random perturbation includes the following steps: Updating the agent's position by the following formula: R(t) = R opt +(R avg -R prev )·L(Dim)·f(t); where, Where R(t) is the position of the agent in the tth iteration, R opt is the optimal position of the agent, R avg is the average value of the best position, R prev is the agent's previous best position, L(Dim) is the Levy flight function, f(t) represents the transition factor function, c is a random number between [0,1], ζ is a random number between [0,1], T is the total number of iterations, s and β are constants, σ ​​is the factor function, μ is a random number between [0,1], and v is a random number between [0,1].

[0090] Then, randomly pair the individuals in the population, perform arithmetic crossover operations on the paired individuals to generate new individuals, and calculate the positions of the new individuals. It should be noted that in the later stages of the algorithm, frozen individuals tend to gather around the optimal solution, resulting in a loss of population diversity. This may cause the algorithm to converge prematurely and fall into a local optimal state. In order to circumvent this special situation, the information exchange between paths is promoted by simulating the weaving and fusion during crystal growth. In path planning, the paths in the current population can be randomly paired, and new paths can be generated through arithmetic crossover operations. This operation helps to retain the characteristics of excellent paths while avoiding falling into local optimality. At the beginning of the algorithm, the individual frost crystals within the population are randomly paired, and each pair is regarded as two nodes in the lattice structure. The algorithm simulates the interconnection and information exchange process between nodes during the growth of the grid, and performs a random pairing of the two original individuals RM i1 and RM i2 Perform arithmetic crossover operation. Two original individuals RM i1 and RM i2 Just like weaving new lattice connections, new individuals are generated and

[0091] Expressed as:

[0092] Expressed as:

[0093] RM i1j and RM i2j Represents RM i1 ,RM i2 The jth dimension (j=1,2,…,D), and Represents RM i1j The j+1th dimension and RM i2j The j+1th dimension of the second generation is generated by the lattice weaving strategy. α1 and α2 are randomly generated random numbers in the range of (0,1). The ω1 and ω2 values ​​are generated by a uniform distribution of random numbers between -1 and 1. The generated second generation is compared with the first generation to retain individuals with better positions.

[0094] The above strategies can retain excellent frost crystal individuals, reduce unnecessary invalid searches, and prevent the algorithm from converging prematurely and leaving the local optimum.

[0095] Compare the individual's updated fitness value with the pre-update fitness value. Retain the individual with the better fitness value and replace the one with the worse fitness value. For a pair of individuals, if one individual has a better fitness value, then retain it and replace the other individual based on the updated solution.

[0096] Determine whether the maximum number of iterations has been reached or other stopping conditions have been met (such as the change in fitness value is less than a certain threshold, etc.). If the stopping conditions are met, the current optimal solution is output to obtain the flight path; otherwise, return to the diffusion stage and continue iteration.

[0097] The tower images are acquired by drones, and the tower images include multiple images of the same tower at different times.

[0098] Preprocess the tower image. Due to the processing method of remote sensing images in the low-altitude domain, target detection requires higher accuracy. Remote sensing images are inevitably affected by different factors such as sensor posture, distance, and quality. Therefore, before target detection, remote sensing images from different periods need to be preprocessed with corresponding precise geometric correction, atmospheric correction, mutual registration, and other image preprocessing. Doing so can improve the accuracy of target detection. The purpose of image registration is to reduce errors caused by differences in shooting angles, etc., to avoid the inability to correspond to the same spatial positions in different images in sequence, and to avoid analyzing the feature points of one image with the feature points at other positions in another image when comparing the feature points extracted later, which can easily lead to a high false detection rate in the target detection results.

[0099] The preprocessed tower image is input into the detection network.

[0100] Reference Figure 5 ,The backbone network of the detection network consists of four stages. Each stage extracts global features and local features, and fuses them.

[0101] For step S300, the detection network constructs a dual-path feature extraction framework. Global features are extracted through a lightweight Transformer network, while local features are extracted through a reversible neural network layer. Feature differentiation is performed on global and local features through element-by-element subtraction followed by an absolute value operation. The lightweight attention inversion module serves as the basic unit for feature extraction. By simplifying the feedforward network structure, the lightweight attention inversion module effectively reduces the embedding dimension and the number of parameters while maintaining comparable performance. Given the importance of detailed edge and texture features in image fusion tasks, the reversible neural network module enhances the retention of input information by achieving bidirectional generation of input and output features.

[0102] Extracting global features of the tower image, including: extracting global features through a lightweight Transformer network.

[0103] Extracting local features of the tower image, including: extracting local features through a lightweight attention inversion module.

[0104] Reference Figure 6 The lightweight attention inversion module includes a local perception module, a lightweight multi-head attention module, and an inverted residual feedforward network. This enhances local feature extraction capabilities while reducing computational complexity and parameters. This helps achieve a lightweight UAV inspection system.

[0105] In the local perception module, the input of the local perception module is added to the result of deep convolution of the input of the local perception module, and the addition result is layer normalized to obtain the output of the local perception module. The output of the local perception module serves as the input of the lightweight multi-head attention module.

[0106] In the lightweight multi-head attention module, the input of the lightweight multi-head attention module is subjected to the first linear mapping result of linear mapping as the query of the multi-head attention mechanism, the input of the lightweight multi-head attention module is subjected to the kxk depth convolution with a step size of k and the second linear mapping result of linear mapping as the key of the multi-head attention mechanism, the input of the lightweight multi-head attention module is subjected to the kxk depth convolution with a step size of k and the third linear mapping result of linear mapping as the value of the multi-head attention mechanism, and the attention feature is obtained according to the query, key and value through the multi-head attention mechanism. The attention feature is normalized by the layer to obtain the output of the lightweight multi-head attention module, and the output of the lightweight multi-head attention module is used as the input of the inverted residual feedforward network.

[0107] In the inverted residual feedforward network, the input of the inverted residual feedforward network is convolved through multiple convolutional layers to obtain the output of the inverted residual feedforward network.

[0108] To maintain translation invariance during data augmentation and focus on local correlations and structural information, a local perception module is inserted before the multi-head attention mechanism. This local perception module extracts local information from the input image through deep convolutions and residual connections, maintaining translation invariance during data augmentation while focusing on local correlations and structural information. Deep convolution reduces the number of parameters and computational complexity compared to traditional convolutions, helping to achieve the goal of lightweight algorithms. This lightweight design is particularly important for drone inspection systems, as drones are often equipped with embedded devices that have limited computing power and storage resources. By reducing the number of model parameters and computational complexity, lightweight algorithms can significantly improve drones' endurance and real-time processing capabilities, enabling them to operate efficiently in complex environments, thereby improving inspection efficiency and reliability.

[0109] In the multi-head self-attention module, in order to save computational cost, a depthwise convolution with a stride of k is used to reduce the size of the key and value. In addition, the dimensions of the query and key are halved. Specifically, in this module, the input X∈R is first transformed into n×d Projection to query key Sum Then, use depthwise convolution to reduce the dimensionality of keys and values ​​to obtain lightweight keys and lightweight value In addition, the LWT block also uses a relative position bias B, which can be learned in each self-attention module.

[0110] The implementation of the lightweight multi-head self-attention module is as follows:

[0111] Among them, bias is randomly initialized. By combining relative position bias, multi-head self-attention can better learn the local correlation of the image. Finally, each head outputs a size of where h is the number of heads, these sequences are concatenated into a final sequence of size n×d.

[0112] To better perform feature extraction and improve performance, the feedforward network in ViT is repositioned within the inverted residual feedforward network. The feedforward network consists of a single activation layer followed by batch normalization. The intermediate convolutions are depthwise, capturing regional features with minimal overhead. Furthermore, a shortcut connection is used between the first and last 1×1 convolutions to improve propagation.

[0113] The formula for the feedforward network is as follows:

[0114] Here, Act(·) represents an activation layer, followed by batch normalization. The convolution in the middle is depthwise, used to capture regional features. In addition, shortcut connections are used to improve propagation capabilities.

[0115] Through the synergistic effect of the local perception module, the lightweight multi-head attention module, and the inverted residual feedforward network, the lightweight attention inversion module effectively extracts and fuses local and global features while maintaining efficient computing, significantly improving the performance of the model in detection tasks and successfully achieving lightweight algorithms. This lightweight design is crucial for drone inspection systems because it not only reduces the consumption of computing resources, but also reduces energy consumption, thereby extending the drone's flight time and enhancing its real-time processing capabilities. This means that drones can cover larger inspection areas over longer periods of time, while reducing dependence on high-end hardware, reducing costs, and improving system reliability and deployability. In addition, efficient computing efficiency helps to quickly detect subtle deformations of towers.

[0116] Reference Figure 3 , for step S400, identifying key features of global features and local features through frequency channel attention and frequency space attention includes the following steps:

[0117] Step S410, performing aggregation in the channel dimension through global pooling to obtain a first aggregated feature;

[0118] Step S420, separating the response components of different channels to the key areas in the aggregated features;

[0119] Step S430, extracting a first low-frequency feature and a first high-frequency feature according to the response component, wherein the frequency of the first low-frequency feature is lower than the frequency of the first high-frequency feature;

[0120] Step S440, performing convolution on the first low-frequency feature and the first high-frequency feature to obtain channel weights of each channel;

[0121] Step S450, performing aggregation in the spatial dimension through global pooling to obtain a second aggregated feature;

[0122] Step S460, performing convolution and residual connection on the second aggregated features to obtain position information;

[0123] Step S470: Perform fast Fourier transform and inverse Fourier transform on the second aggregated feature to obtain a second low-frequency feature and a second high-frequency feature, wherein the frequency of the second low-frequency feature is less than the frequency of the second high-frequency feature;

[0124] Step S480, convolving the second low-frequency feature and the second high-frequency feature to obtain a spatial weight of each space;

[0125] Step S490: identifying and obtaining key features based on the channel weights and spatial weights.

[0126] The frequency channel module executes steps S410, S420, S430, and S440 to obtain the channel weight of each channel; the frequency space module executes steps S450, S460, S470, and S480 to obtain the spatial weight of each space.

[0127] In the frequency channel module, channel information is aggregated through global pooling in the channel dimension. The aggregation process is expressed as: Among them, y represents the aggregated channel features, Cat(·) is the operation of splicing tensors along the specified dimension, q1 and q2 represent the dual-phase features processed by the frequency channel module, AVg(·) and Max(·) correspond to the global average pooling and global maximum pooling operations in the channel dimension, Ca(·) is the frequency domain channel module, T(·) represents the Transformer encoder, I(·) represents the invertible neural network (INN), t represents the global feature, and i represents the local feature.

[0128] Traditional attention mechanisms often struggle to fully preserve the frequency domain components of multi-source features. To address this limitation and maintain complementary information, a frequency channel module is employed to separate the response components of different channels to key regions from a frequency domain analysis perspective. Its mathematical representation is: Ca(t) = σ(FC(ReLU(FC(DCT(t)))))); where σ(·) represents the sigmoid activation function, FC(·) denotes the fully connected layer operation, and DCT(·) corresponds to the two-dimensional discrete cosine transform. The discrete cosine transform (DCT) converts image data from the time domain to the frequency domain and expresses signal characteristics through linear combinations of the data.

[0129] Two-dimensional discrete cosine transform is used to extract low-frequency features because it can concentrate the main energy on low-frequency components during the signal frequency domain conversion process, while high-frequency components usually have smaller amplitude characteristics. The two-dimensional discrete cosine transform process is expressed as: Among them, h∈{0,1,…,H-1},w∈{0,1,…,W-1}.

[0130] The channel features are processed through two one-dimensional convolutional layers to calculate the weight distribution of each channel in the feature. These weights are normalized by the Softmax function so that their sum is 1, thus enabling comparable analysis of the weights of the bi-temporal features. By comparing the weight values, channels with higher significance can be identified, thereby highlighting the most critical information components in the bi-temporal channel features. The generated channel weights can be expressed as: w c1 ,w c2 =σ(ECA(y)); where σ(·) represents the sigmoid activation function, w c1 and w c2They represent the channel weights of the two time phases respectively, and ECA(·) represents the one-dimensional convolution operation.

[0131] Specifically, the input features are processed by 2D discrete cosine transform, fully connected layer, RELU activation function, fully connected layer and sigmoid activation function, and the input features are jump-connected to the processing results to multiply the input features and the processing results to obtain the output features.

[0132] In the frequency-space module, global features and spatial information are integrated through global pooling operations in the spatial dimension, which can be expressed as follows: Here, z represents the aggregated spatial features and Sa(·) is the frequency-space module.

[0133] To enhance the fusion of spatial and frequency information, a frequency-space module, denoted as Sa(·), is designed. This module is specifically designed to extract and integrate features from the spatial and frequency domains to provide a more comprehensive understanding of the data. The introduction of this module not only optimizes feature aggregation but also reduces the spatial and temporal complexity of the processing.

[0134] The frequency-space module can be expressed as: Among them, σ(·) is the sigmoid activation function, Conv 1×1 (·) denotes a 1×1 convolution operation, and W is a learnable weighted gating mechanism whose parameter setting is related to the width and height dimensions of the input image.

[0135] The frequency-space module consists of two core components: one is the position module, which accurately captures position details through residual connections and convolution to improve features. The other is the frequency domain module, which consists of a fast Fourier transform layer and an inverse Fourier transform layer. By introducing frequency components to extract different information, it forms a multispectral description and fully utilizes the information. The fast Fourier transform layer is used to extract high-frequency features. The fast Fourier transform layer decomposes the signal into different frequency components and introduces a learnable weighted gating W to select useful high-frequency and low-frequency components. Specifically, in one branch, the input features are fast Fourier transformed, and useful high-frequency and low-frequency components are selected through weighted gating, and then an inverse fast Fourier transform is performed. In the other branch, the input features are subjected to 1x1 convolution and sigmoid activation function operations. The input features and the operation results are then jump-connected to multiply the two results. The results of the two branches are added to obtain the output features.

[0136] The generated spatial weight can be expressed as: w s1 ,w s2 =σ(Conv(z)); where σ(·) represents the sigmoid function, w s1 and w s2represents the spatial weight of the bi-temporal layer, and Conv(·) represents a 1x1 convolution.

[0137] At each stage, a dual-phase image (two images of the same tower taken at different times) is input into the LWT module. The LWT module performs element-wise subtraction followed by an absolute value operation on the two output results of the dual-phase image to perform feature differentiation on the global and local features. The dual-phase image is input into the reversible neural network, and the two output results are subjected to element-wise subtraction followed by an absolute value operation to perform feature differentiation on the global and local features. The two differential results are respectively input into the frequency channel module and the frequency space module to calculate the channel weight and spatial weight. The results of the frequency channel module and the frequency space module are average pooled and maximum pooled. The pooled results in the channel dimension are convolved through the convolution module. The pooled results in the spatial dimension are pooled through the efficient channel attention module to extract attention features. The convolution result and the attention feature are fused, and the fused feature is input into the step-by-step modification detection module.

[0138] The channel and spatial weights of the bi-temporal features are fused to create a comprehensive set of bi-temporal weights, thereby highlighting the key parts of the bi-temporal features. These weights are then used to multiply and combine the bi-temporal features to achieve seamless fusion of temporal information. The final output is expressed as: r = (w c1 +w s1 )*t+(w c2 +w s2 )*i; where r represents the fused feature. This process prioritizes the key elements in the bi-temporal features while filtering out the less important ones, thus achieving more refined and effective feature fusion.

[0139] For step S500, the detection module is gradually modified to perform dilated convolution operations with different dilation rates on the key features to obtain multi-scale features.

[0140] Reference Figure 4 , where the dilated convolution operation includes the following steps:

[0141] Step S510, performing average pooling, group convolution and classification on the input of the dilated convolution layer to obtain classification features;

[0142] Step S520, performing group convolution on the input of the dilated convolution layer to obtain group convolution features;

[0143] In step S530, the result of multiplying the classification feature and the grouped convolution feature is reshaped and convolved to obtain the output of the dilated convolution layer.

[0144] The performance of convolutional neural networks is hampered by the limitations of traditional convolution operations, which rely on shared parameters and are insensitive to information differences caused by positional variations. Therefore, by applying dilated convolutions at varying dilation rates to key features, we can capture feature information at different scales, which is crucial for drones detecting objects and recognizing small deformations in complex environments. Secondly, dilated group convolutions, through grouped convolutions, reduce the number of parameters and computational complexity while enhancing the model's ability to extract local features. This characteristic makes dilated group convolutions particularly effective in lightweight models used on drones.

[0145] Reference Figure 7 In the gradual modification detection module, the input feature r is subjected to a dilated convolution with a dilation rate of 5 (d=5), the dilated convolution result and the input feature are element-wise multiplied, the multiplication result is activated by the ReLU activation function, the activation result is subjected to a dilated convolution with a dilation rate of 3 (d=3), the dilated convolution result and the input feature are element-wise added, the addition result is activated by the ReLU activation function, the activation result is subjected to a dilated convolution with a dilation rate of 1 (d=1), the dilated convolution result and the input feature are element-wise added, and the addition result is activated by the ReLU activation function to obtain the output feature u.

[0146] Dilated group convolutions address these issues by prioritizing the importance of different features within a receptive field and emphasizing the spatial properties of that field. As a result, the feature maps generated by dilated group convolutions are better at capturing spatial details because the learned attention mechanism is tailored to each specific receptive field, avoiding the previous problem of sharing feature information across fields. Due to the use of grouped convolutions, the height and width of the reshaped features are tripled, and 3×3 dilated convolutions with different dilation rates are used to further extract feature information.

[0147] The formula for dilated group convolution is expressed as: F d (r)=Conv d (Softmax(Convg 1×1 (Avg(r)))*ReLU(Norm(Convg 3×3 (r)))); where d = 1, 3, 5, Convg 1×1 (·) represents 1×1 group convolution, Convg 3×3 (·) represents 3×3 group convolution, Norm(·) represents normalization operation, x is the input feature map, Conv d (·) represents a dilated convolution with a dilation rate of d.

[0148] The progressively modified detection module (SMDM) is used to better capture time-varying information and improve the integrity of features within the changing region. This module can simultaneously consider information from multiple receptive fields and learn multi-scale features through three dilation group convolution modules with different dilation rates and residual connections.

[0149] SMDM is defined by the following formula: r' = ReLU (r + F d (r)), d = 5; r' = ReLU (r + F d (r')), d = 3; u = ReLU (r + F d (r")), d = 1; where r represents the input of SMDM and u represents the output of SMDM.

[0150] By adopting a ladder-like structure, SMDM systematically explores temporal changes at different receptive field sizes, from larger to smaller ones. This ensures that the interconnected branches enhance each other’s effectiveness in detecting these changes.

[0151] In step S600, supervised learning is performed based on the multi-scale features through the supervised learning module to integrate semantic information and geometric information to generate supervised attention features.

[0152] In traditional layer-by-layer feature extraction, high-level networks excel at capturing semantic information, but the resulting feature maps are low-resolution, limiting their ability to represent geometric details. Conversely, low-level networks, thanks to their high-resolution feature maps, excel at capturing geometric details but are less effective at encoding semantic information. The supervised learning module refines and enhances features, enabling more effective multi-level integration. The supervised learning module effectively integrates high-level semantic information with low-level geometric details, facilitating drones' target detection and subtle deformation recognition in complex environments. Furthermore, through feature refinement and enhancement, the supervised learning module increases the model's focus on key features, reduces noise aggregated across multiple layers, and eliminates irrelevant information. This makes the supervised learning module exceptionally effective in lightweight drone-mounted models, particularly when processing high-resolution imagery, highlighting key features and filtering out unimportant information.

[0153] Reference Figure 8 In one branch of the supervised learning module, the input feature u passes through the fully connected layer and the RELU activation function; in the other branch of the supervised learning module, the input feature passes through the fully connected layer, the output of the fully connected layer passes through a 1x1 convolution, the convolution result is inverted, the convolution result and the inversion result are connected, the connection result passes through the Sigmoid activation function, the results of the two branches are element-wise multiplied, and the multiplication result passes through a 3x3 convolution to obtain the output feature m.

[0154] Specifically, fully connected layers and ReLU activation layers are added to condense spatial information into a dense representation. This not only improves the model's focus on key features, but also reduces noise from multi-layer aggregation and eliminates irrelevant information. Fully connected layers and convolutions are applied, followed by a sigmoid activation function to generate features. The inverse change map is then calculated through the reverse process. This allows for the representation of changing regions while capturing stable regions. Finally, the sums are concatenated and passed through a convolutional layer to generate the attention mask d.

[0155] The supervised learning module can be expressed as follows: Here, u is the input feature, σ(·) represents the sigmoid function, and Cat(·) is an operation that concatenates two tensors along a specified dimension.

[0156] By generating attention-weighted features through a, the features are refined. This process is expressed as: m = Conv 33 (ReLU(FC(u))*a); a acts as a filtering mechanism on the input u, retaining only features related to the change. Finally, the supervised learning module generates supervised attention to guide the further fusion of feature m with higher-level features.

[0157] In step S700, the supervised attention features are decoded by a decoder to generate a change segmentation map, and target detection is performed based on the change segmentation map to obtain a tower deformation detection result.

[0158] The decoder consists of four LWT layers that correspond to the layers in the encoder and use the same number and embedding dimensions in reverse order. Each layer of the decoder contains the same number of LWT blocks as the encoder and has an upsampling module that gradually restores the original size of the image through convolution operations and integrates features of different scales through skip connections.

[0159] The decoder input is a multi-scale feature map processed by the fusion module. In each decoder layer, the input feature map is first processed by the LWT block and then fed into the patch merging module, which involves linear upsampling to double the size and convolution to halve the number of channels. It is then summed with the fused feature map of the same size and dimensions. This allows the decoder to extract multi-scale variation information for generating the final variation segmentation map.

[0160] The two sets of images taken by the drone are pre-processed and then input into the trained target detection network. The segmentation map of the changes is then output and transmitted to the staff's relevant applications via the Internet. Finally, the staff will further determine whether there are any post-disaster tower changes in the corresponding area.

[0161] In the process of training the detection network, a loss function is obtained according to the tower deformation detection results; the parameters of the detection network are adjusted according to the loss function; among them, the loss function of the detection network is the sum of the classification loss and the weighted cross entropy loss.

[0162] A hybrid loss function is defined. The model uses a hybrid loss function that combines classification loss and weighted cross entropy loss to optimize model performance. The loss function is defined as: L = λ·L wce +(1-λ)·L dice ; where λ is the weight parameter, L wce and L dice They represent weighted cross entropy loss and classification Dice loss respectively.

[0163] L wce Expressed as:

[0164] L dice Expressed as:

[0165] L wce To solve the problem of class imbalance, L dice Used to evaluate the similarity between the predicted image and the true image.

[0166] The parameters of the detection network are adjusted through the loss function to improve the accuracy of the detection network in detecting tower deformation.

[0167] An embodiment of the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-described drone-based tower deformation detection method when executing the computer program.

[0168] The electronic device may be any intelligent terminal including a computer.

[0169] In general, for the hardware structure of the electronic device, the processor can be implemented using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0170] The memory can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory and is called by the processor to execute the methods of the embodiments of this application.

[0171] The input / output interface is used to realize information input and output.

[0172] The communication interface is used to realize the communication interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0173] The bus transmits information between the various components of the device (such as the processor, memory, input / output interface, and communication interface). The processor, memory, input / output interface, and communication interface communicate with each other within the device through the bus.

[0174] An embodiment of the present application provides a computer storage medium storing computer-executable instructions for executing the above-mentioned method for detecting tower deformation based on a drone.

[0175] Those skilled in the art will appreciate that all or some of the steps and systems in the method disclosed above can be implemented as software, firmware, hardware, and appropriate combinations thereof. Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, and the computer-readable medium can include computer storage media (or non-transitory media) and communication media (or temporary media). As known to those skilled in the art, the term computer storage media is included in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data) and is volatile and non-volatile, removable, and non-removable. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory, or other memory technology, CD-ROM, digital versatile disks (DVD), or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage, or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media generally contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and may include any information delivery medium. In the above description of this specification, the reference terms "one embodiment / example", "another embodiment / example" or "certain embodiments / examples" and the like are intended to mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0176] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0177] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0178] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0179] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0180] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the above-mentioned units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. Although the embodiments of the present application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and purpose of the present application, and the scope of the present application is defined by the claims and their equivalents.

[0181] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present application, and these equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A tower deformation detection method based on drone, characterized in that: include: Acquire tower images through a drone, and input the tower images into the detection network, wherein the tower images include multiple images of the same tower at different times; Extracting global features and local features of the tower image; Identifying key features of the global features and the local features through frequency channel attention and frequency space attention; Performing dilated convolution operations with different dilation rates on the key features to obtain multi-scale features; Performing supervised learning based on the multi-scale features to integrate semantic information and geometric information to generate supervised attention features; The supervised attention feature is decoded to generate a change segmentation map, and target detection is performed based on the change segmentation map to obtain a tower deformation detection result.

2. The tower deformation detection method based on drone according to claim 1 is characterized in that: Follow these steps to plan the flight path of said drone: Initializing a population, wherein the population includes a plurality of agents, and the agents are composed of a plurality of individuals; Calculate the fitness value of individuals in the population; Update the frost factor and exploration factor based on the current number of iterations; When the frost factor is less than a preset adhesion coefficient, the agent's position is moved toward the current optimal solution; When the frost factor is greater than or equal to a preset adhesion coefficient, the position of the agent is updated by random perturbation according to the numerical relationship between the fitness value and the exploration factor; Randomly pair individuals in the population, perform arithmetic crossover operations on the paired individuals to generate new individuals, and calculate the positions of the new individuals; Replace individuals according to their fitness values ​​before and after update; Output the optimal path based on the position of the replaced individual.

3. The tower deformation detection method based on drone according to claim 1 is characterized in that: The updating of the agent's position by random perturbation comprises: Update the agent's position by: R(t) = R opt +(R avg -R prev )·L(Dim)·f(t); where, Where R(t) is the position of the agent in the tth iteration, R opt is the optimal position of the agent, R avg is the average value of the best position, R prev is the agent's previous best position, L(Dim) is the Levy flight function, f(t) represents the transition factor function, c is a random number between [0,1], ζ is a random number between [0,1], T is the total number of iterations, s and β are constants, σ ​​is the factor function, μ is a random number between [0,1], and v is a random number between [0,1].

4. The tower deformation detection method based on drone according to claim 1 is characterized in that: Extracting global features of the tower image includes: extracting global features through a lightweight Transformer network.

5. The tower deformation detection method based on drone according to claim 1 is characterized in that: Extracting local features of the tower image, including: extracting local features through a lightweight attention inversion module; The lightweight attention inversion module includes a local perception module, a lightweight multi-head attention module and an inverted residual feedforward network; In the local perception module, the input of the local perception module is added to the result of deep convolution of the input of the local perception module, and the addition result is layer normalized to obtain the output of the local perception module. The output of the local perception module serves as the input of the lightweight multi-head attention module; In the lightweight multi-head attention module, the input of the lightweight multi-head attention module is subjected to a first linear mapping result of linear mapping as a query of the multi-head attention mechanism, the input of the lightweight multi-head attention module is subjected to a second linear mapping result of depth convolution and linear mapping as a key of the multi-head attention mechanism, the input of the lightweight multi-head attention module is subjected to a third linear mapping result of depth convolution and linear mapping as a value of the multi-head attention mechanism, and the attention feature is obtained according to the query, key and value through the multi-head attention mechanism, the attention feature is normalized by the layer to obtain the output of the lightweight multi-head attention module, and the output of the lightweight multi-head attention module is used as the input of the inverted residual feedforward network; In the inverted residual feedforward network, the input of the inverted residual feedforward network is convolved through multiple convolutional layers to obtain the output of the inverted residual feedforward network.

6. The tower deformation detection method based on drone according to claim 1, characterized in that: The identifying key features of the global features and the local features through frequency channel attention and frequency space attention includes: Aggregate through global pooling in the channel dimension to obtain the first aggregated feature; separating response components of different channels to the key regions in the aggregated features; extracting a first low-frequency feature and a first high-frequency feature according to the response component, wherein the frequency of the first low-frequency feature is lower than the frequency of the first high-frequency feature; Convolving the first low-frequency feature and the first high-frequency feature to obtain a channel weight of each channel; Aggregate through global pooling in the spatial dimension to obtain the second aggregated feature; Performing convolution and residual connection on the second aggregated features to obtain position information; Performing a fast Fourier transform and an inverse Fourier transform on the second aggregated feature to obtain a second low-frequency feature and a second high-frequency feature, wherein the frequency of the second low-frequency feature is less than the frequency of the second high-frequency feature; Convolving the second low-frequency feature and the second high-frequency feature to obtain a spatial weight of each space; Key features are identified based on the channel weights and the spatial weights.

7. The tower deformation detection method based on drone according to claim 1, characterized in that: The dilated convolution operation includes: The input of the dilated convolutional layer is average pooled, grouped, and classified to obtain classification features; Perform group convolution on the input of the dilated convolution layer to obtain group convolution features; The result of multiplying the classification feature and the grouped convolution feature is reshaped and convolved to obtain the output of the dilated convolution layer.

8. The tower deformation detection method based on drone according to claim 1, characterized in that: The method further comprises: Obtaining a loss function according to the tower deformation detection result; Adjusting parameters of the detection network according to the loss function; The loss function of the detection network is the sum of the classification loss and the weighted cross entropy loss.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for detecting tower deformation based on a drone as claimed in any one of claims 1 to 8 is implemented.

10. A computer storage medium, characterized in that Computer-executable instructions are stored, and the computer-executable instructions are used to execute the tower deformation detection method based on a drone as described in any one of claims 1 to 8.

Images