Operation method of high-rise outer wall disease identification system based on unmanned aerial vehicle

Abstract

The invention discloses an operation method of a high-rise outer wall disease identification system based on an unmanned aerial vehicle. The method belongs to the technical field of image detection, and comprises the following implementation steps: completing flight path planning through an unmanned aerial vehicle carrying an RGB-D camera, and photographing, identifying and classifying diseases of a high-rise outer wall; according to the flight path planning process of the unmanned aerial vehicle, the unmanned aerial vehicle carries an RGB-D camera to complete establishment of a high-rise environment model, and an improved HLOA is adopted to complete flight path planning of the unmanned aerial vehicle so as to obtain the shortest and safest flight path of the unmanned aerial vehicle; a camera is carried in the flight path process of the unmanned aerial vehicle to collect the high-rise outer wall disease information, the high-rise outer wall crack and crack disease image information is classified by adopting the novel algorithm MUME provided by the invention, and the recognition and detection of the high-rise outer wall disease information can be completed more quickly and more accurately by adopting the novel MUME algorithm provided by the invention.

Description

Technical Field

[0001] The invention belongs to the technical field of image detection and relates to an operation method of a high-rise building exterior wall disease recognition system based on an unmanned aerial vehicle. Background Art

[0002] High-rise buildings are important infrastructure for people's lives; the emergence of high-rise buildings has not only changed people's living conditions, but also brought a series of challenges and problems; due to long-term use and natural disasters, various defects such as cracks and fissures may appear on the surface of high-rise buildings; these defects, if not repaired, maintained and managed in time, will seriously affect the safety and stability of high-rise buildings, and may even cause disasters and accidents.

[0003] At present, the identification and detection of surface defects of high-rise buildings mainly rely on manual inspection and visual inspection; however, manual inspection faces limitations in terms of manpower, material resources, time, etc., and the accuracy and reliability of visual inspection also have certain limitations, which hinder the discovery and treatment of surface defects of high-rise buildings; therefore, the development of an efficient, accurate, reliable and intelligent method for identifying surface defects of high-rise buildings will provide strong guarantees for ensuring the safety and stability of high-rise buildings and reducing the occurrence of safety accidents. Summary of the invention

[0004] In view of the above problems, the purpose of the present invention is to propose an operation method of a high-rise building exterior wall defect identification system based on a drone, aiming to improve the recognition accuracy of high-rise building defect phenomena.

[0005] The technical solution of the present invention is: the operation method of the high-rise building exterior wall disease identification system based on the drone of the present invention completes the track planning by carrying an RGB-D camera on the drone, and then completes the identification and classification of the high-rise building exterior wall disease phenomenon by carrying an RGB-D camera, and the operation steps are as follows:

[0006] Step (1): The drone uses the laser radar and RGB-D camera to scan and model the high-rise environment, obtain point cloud data, and set target points;

[0007] Step (2): In the process of UAV path planning, the improved horned lizard algorithm (HLOA) is used for path planning, in which the UAV trajectory is constrained by using the path smoothing method and constraint function to obtain the shortest and safest UAV flight trajectory;

[0008] Step (3): When the drone flies along the planned path, every time it reaches a target point, it uses the onboard RGB-D camera to photograph the exterior wall defects of the high-rise building, and uses the novel MUME algorithm to identify and classify the exterior wall defects of the high-rise building.

[0009] Furthermore, in step (1), the specific method for the laser radar and RGB-D camera to collect high-rise building environment information is:

[0010] Define the m target point set for acquisition settings based on the LiDAR and RGB-D camera: T = t 1 ,t 2 ,...,t m and the initial position of the drone: S = (x 0 ,y 0 ,z 0 );

[0011] Define the path planning sequence of the drone based on the laser radar: P = p 1 ,p 2 ,...,p m+1 , where p i Indicates the location of the drone at step i.

[0012] Furthermore, in step (2), the improved horned lizard algorithm (HLOA) is used for path planning, and the specific method is as follows:

[0013] First, the initial population (search agent) is defined as is the i-th horned lizard in the t-th generation (time); the Logistic mapping of chaotic mapping is introduced, and its randomness and ergodicity are used to improve the search ability, and the fitness of each horned lizard is calculated at the same time;

[0014] The mathematical expression of Logistic mapping is expressed as:

[0015] Z i (t) = μZ i (t-1)(1-Z i (t-1))

[0016] Where t is the iteration time, x(t)∈[0,1], μ is an adjustable parameter, μ∈[0,4]; Z(t) is the proportion of the population at time t to the maximum possible population size (i.e., the ratio of the current population size to the maximum possible population size), and in this method, μ is 4;

[0017] The horned lizard individuals added to the logistic map are updated by the following formula:

[0018]

[0019] Make the initial population evenly distributed in the space and make the obtained initial population closer to the optimal solution;

[0020] When a horned lizard detects danger, it will choose different strategies according to the situation, first creating a "hidden strategy"; definition is the i-th search agent in the t+1-th generation space, is the search agent with the best fitness in the tth generation, and the calculation formula is:

[0021]

[0022] In the formula, r 1 、r 2 、r 3 and r 4 is a random integer between 1 and the maximum number of search agents that satisfies r 1 ≠r 2 ≠r 3 ≠r 4 ; Max_iter represents the maximum number of iterations (generations); σ is a binary number that determines whether the random number from the color table is less than or equal to 0.5. If it is satisfied, it is 1, otherwise it is 0; Set to 2, c 1 and c 2 is the simulated chromaticity and c 1 ≠c 2 ;

[0023] The improved HLOA abandons the "attack strategy" with low efficiency in path planning, and chooses to create a "mobile escape strategy" to improve the safety of the drone's handling of obstacles. The formula is:

[0024]

[0025] Where walk is a random number generated between -1 and 1; ε is a random number generated by the standard Cauchy distribution, where σ is set to 0 and 1;

[0026] The worst search agent is replaced by the "skin lightening and darkening strategy", the formula is:

[0027]

[0028] In the formula, is the search agent with the worst fitness in the tth generation; Light 1 and Light 2 is a random number generated between 0 and 0.4046661; Dark 1 and Dark 2 is a random number generated between 0.5440510 and 1;

[0029] In order to avoid the algorithm from falling into the problem of premature convergence and jumping out of the local optimum, a random difference mutation strategy is introduced to use the position of the current best individual and the position of the randomly selected individual to perform random difference processing to generate a new position; the specific formula is as follows:

[0030]

[0031] In the formula, r 5 Represents a random number in the range [0,1]; represents the latest individual in generation t+1; represents a random individual in the tth generation;

[0032] The planned path is smoothed and the B-spline algorithm is used to process the redundant transformations and corners in the path. The B-spline curve is:

[0033]

[0034] Where P i are the vertices of the characteristic polygon, which are given by path(P 0 ,P 1 ,...) is used to construct the point type of the spline, B i,k is the k-order (k-1) basis function, and after substituting it into the path, we get:

[0035]

[0036] B i,1 (u)=1,u i <u<u i+1 ; B i,k (u)=0,Otherwise.

[0037] Set the UAV constraint function, introduce the performance constraints of the UAV's yaw angle and pitch angle, filter out some subsequent nodes, reduce the calculation complexity, and improve the fit and real-time performance of path planning;

[0038] Let the current node of the drone be X i , the next node selected by the drone is X i+1 , the previous parent node is X i-1 , its heading angle φ(X i ,X i+1 ) and pitch angle The formula is as follows:

[0039]

[0040] In the formula, (x i ,y i ,z i ) is node X i The three-dimensional coordinates of (x i+1 ,y i+1 ,z i+1 ) is the next node X i+1 The three-dimensional coordinates of (x i-1 ,yi-1 ,z i-1 ) is the previous parent node X i-1 The three-dimensional coordinates of

[0041]

[0042] The restriction set of the drone when selecting nodes is expressed as:

[0043]

[0044] Where N is point X i The set of all successor nodes of max and Indicates the maximum heading and pitch angles affected by the drone's performance.

[0045] Furthermore, in step (3), the specific method of using the novel MUME algorithm to identify and classify high-rise building exterior wall disease phenomena is:

[0046] The drone flies along the optimal path output by the improved HLOA. Whenever the drone reaches a target point, it uses the onboard RGB-D camera to photograph the exterior wall of the high-rise building. Based on the collected images, a dataset of image defects on the exterior wall of the high-rise building is established. The defects on the exterior wall of the high-rise building mainly include horizontal cracks and vertical cracks.

[0047] The joint MUME algorithm is used to identify and classify the defects of high-rise building exterior walls. The recognition process of the joint MUME algorithm is to perform image enhancement on the collected images of high-rise building exterior wall defects using the MSRCR algorithm, obtain accurate crack segmentation maps through the improved U-Net model and perform binarization, and then use morphological filtering to denoise and smooth the binary image of the cracks. Finally, the mathematical model of the EfficientNetv2-AttnConv network is constructed to classify and identify cracks and fissures in high-rise building defects.

[0048] The MSRCR algorithm performs image enhancement and multi-scale color enhancement. The MSRCR algorithm performs multi-scale decomposition on the image, performs Retinex algorithm enhancement on each scale image, obtains reflectance and brightness components, and then performs color correction and weighted synthesis. The entire algorithm has low computational complexity and is relatively fast. When the MSRCR algorithm uses the Retinex algorithm to enhance images of different scales, it can obtain better enhancement effects, and can keep the color of the image unchanged during the enhancement process, without causing oversaturation or color cast problems due to enhancement. The formula of MSRCR is as follows:

[0049]

[0050] C i(x,y)=β(log(αI i (x,y))-log(∑ i∈{r,g,b} I i (x,y)))

[0051] In the formula, β represents the gain value, and the size of α is related to the strength of nonlinearity;

[0052] The improved U-Net model is used to obtain accurate crack segmentation maps and perform binarization. The multi-scale feature prediction fusion neural network is used to improve the U-Net network. The improved U-Net network retains the original convolution kernel pooling part and applies this fusion prediction to the features of the middle layer of the U-Net network. This method can improve the network's receptive field, obtain contextual information in different situations, and enhance the network's ability to understand and judge target features. The output formula of the multi-scale feature prediction fusion is expressed as:

[0053] Z(x)=Conv([Z 0 ,Z 1 ,Z 2 ,Z 3 ,Z 4 ])

[0054] In the formula, Z(x) represents the output after feature prediction fusion, Z 0 ,Z 1 ,Z 2 ,Z 3 ,Z 4 Represented as upsampled output feature maps of different scales.

[0055] The design of loss function has a great impact on the performance of network model. In the past, most image segmentation tasks used cross entropy as the target loss function. The cross entropy loss function is used for pixel-level classification. For crack images, the target cracks account for a small proportion, which will lead to the problem of class imbalance. Therefore, the Dice loss function is introduced. Considering the advantages and disadvantages of the cross entropy loss function and the Dice loss function, the two loss functions are weighted and summed to obtain a new loss function. The new loss function formula is:

[0056]

[0057] L=αL CE +(1-α)L Dice

[0058] Where N is the number of pixels, y is i represents the label of the i-th pixel, Represents the prediction result of the i-th pixel; ε is a hyperparameter to prevent the denominator from being 0; α∈[0,1] is the weight coefficient;

[0059] Finally, the obtained crack segmentation map is binarized.

[0060] Morphological filtering is used to denoise and smooth the crack binary image. Morphological filtering is a nonlinear filtering technology. Its core idea is to construct specific structural elements to perform translation matching or local correction on the original signal from front to back, suppressing noise while retaining the morphological characteristics of the signal. It can effectively eliminate some isolated small points or small connected areas through operations such as corrosion and expansion, and can also eliminate burrs and uneven parts to achieve the purpose of smoothing the image edge.

[0061] The present invention selects the opening and closing operation in morphological processing to filter the binary image; the opening and closing operation is mainly based on the combination of corrosion and expansion operators, and the formula is:

[0062]

[0063] In the formula, A represents the object to be processed; B is the structural element; is the expansion symbol; Θ is the erosion symbol;

[0064] The images of high-rise building defects and cracks are divided into a test set and a training set and input into the EfficientNetv2-A model. The mathematical model of the EfficientNetv2-A network is trained to classify and identify high-rise building defects and cracks.

[0065] The beneficial effects of the present invention are: 1. The present invention takes into account that in the actual environment, the UAV needs to constantly adapt to the high-rise environment; so in the process of UAV path planning, the optimal path will appear to be more complicated; considering the actual environment, the present invention adopts an improved horned lizard algorithm that integrates path smoothing and constraint functions to constrain the UAV track, so as to improve the safety and rationality of UAV flight; 2. The present invention proposes to use a novel MUME algorithm to identify and classify high-rise building exterior wall disease information; the process of the novel MUME algorithm is: the MSRCR algorithm performs image enhancement on the collected high-rise building exterior wall disease image, and an accurate crack segmentation map is obtained and binarized through an improved U-Net model, and then morphological filtering is used to denoise and smooth the crack binarized image, and finally a mathematical model of the EfficientNetv2-AttnConv network is constructed to classify and identify high-rise building disease cracks and cracks. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 is a flowchart of an embodiment of the present invention;

[0067] Figure 2 Flow chart of the novel MUME algorithm in an embodiment of the present invention. DETAILED DESCRIPTION

[0068] The specific technical scheme of the present invention is further described in detail below with reference to specific examples.

[0069] As shown in the figure, the operating method of the high-rise building exterior wall disease identification system based on the drone of the present invention is to scan and model the high-rise building environment through the laser radar and RGB-D camera carried by the drone, complete the path planning through the improved horned lizard algorithm, and then complete the identification and classification of the high-rise building exterior wall disease phenomenon through the carried RGB-D camera; its specific implementation process is:

[0070] S1: The drone uses the laser radar and RGB-D camera to scan and model the high-rise environment, obtain point cloud data and set target points;

[0071] Specifically, the drone is started first, and the aircraft model battery starts to power each module of the drone. The motor drives the drone wings to start working. By controlling the four wings, the drone is driven to reach any coordinate position in the space range. Then the laser radar and RGB-D camera carried by the drone are started to scan and model the high-rise environment, so as to obtain the general shape of the high-rise building, and obtain point cloud data through the depth sensor built into the camera.

[0072] The target point is set according to the collected point cloud data, and the improved HLOA is used to complete the path planning of the UAV. When the UAV moves through all the viewpoint coordinate positions, the high-rise building inspection task is completed.

[0073] S2: In the process of UAV path planning, the improved horned lizard algorithm (HLOA) is used for path planning, in which the path smoothing method and constraint function are used to constrain the UAV trajectory to obtain the shortest and safest UAV flight trajectory;

[0074] Specifically, considering the actual environment, the UAV needs to constantly adapt to the complex environment of high-rise buildings; therefore, in the process of UAV trajectory planning, the optimal path will appear to be more complex and contain many different features, so the improved HLOA algorithm is used to perform UAV path planning;

[0075] Step 1: Define the m target point set for the acquisition setting: T = t 1 ,t 2 ,...,t m and the initial position of the drone: S = (x 0 ,y 0 ); Define the distance matrix between the drone and the target point: D ij =d ij (m+1×m+1), where d ij represents the Euclidean distance from point i to point j;

[0076] Step 2: Define the path planning sequence of the drone: P = p 1 ,p 2 ,...,p m+1 , where p i represents the position of the drone at step i;

[0077] Step 3: Input the definition points and high-rise modeling information into the improved horned lizard algorithm; generate the initial population (search agent) defined as is the i-th horned lizard in the t-th generation (time); the Logistic mapping of chaotic mapping is introduced, and its randomness and ergodicity are used to improve the search ability, and the fitness of each horned lizard is calculated at the same time;

[0078] The mathematical expression of Logistic mapping is expressed as:

[0079] Z i (t) = μZ i (t-1)(1-Z i (t-1))

[0080] Wherein, t is the iteration time, x(t)∈[0,1], μ is an adjustable parameter, μ∈[0,4]; Z(t) is the proportion of the population at time t to the maximum possible population size (i.e., the ratio of the current population size to the maximum possible population size), and μ is 4 in the present invention;

[0081] The horned lizard individuals added to the logistic map are updated by the following formula:

[0082]

[0083] Make the initial population evenly distributed in the space and make the obtained initial population closer to the optimal solution;

[0084] Step 4: When a horned lizard detects danger, it will choose different strategies according to the situation, first creating a "hidden strategy"; define is the i-th search agent in the t+1-th generation space, is the search agent with the best fitness in the tth generation, and the calculation formula is:

[0085]

[0086] In the formula, r 1 、r 2 、r 3 and r 4 is a random integer between 1 and the maximum number of search agents that satisfies r 1 ≠r 2 ≠r 3 ≠r 4; Max_iter represents the maximum number of iterations (generations); σ is a binary number that determines whether the random number from the color table is less than or equal to 0.5. If it is satisfied, it is 1, otherwise it is 0; Set to 2, c 1 and c 2 is the simulated chromaticity and c 1 ≠c 2 ;

[0087] Step 5: Improved HLOA abandons the "attack strategy" with low efficiency in path planning and chooses to create a "mobile escape strategy" to improve the drone's handling of obstacles and improve safety. The formula is:

[0088]

[0089] Where walk is a random number generated between -1 and 1; ε is a random number generated by the standard Cauchy distribution, where σ is set to 0 and 1;

[0090] Step 6: Replace the worst search agent through the "skin lightening and darkening strategy", the formula is:

[0091]

[0092] In the formula, is the search agent with the worst fitness in the tth generation; Light 1 and Light 2 is a random number generated between 0 and 0.4046661; Dark 1 and Dark 2 is a random number generated between 0.5440510 and 1;

[0093] Step 7: In order to avoid the algorithm from falling into the problem of premature convergence and jumping out of the local optimum, a random difference mutation strategy is introduced to perform random difference processing on the position of the current best individual and the position of the randomly selected individual to generate a new position; the specific formula is as follows:

[0094]

[0095] In the formula, r 5 Represents a random number in the range [0,1]; represents the latest individual in generation t+1; represents a random individual in the tth generation;

[0096] Step 8: Smooth the planned path and use the B-spline algorithm to process the redundant transitions and corners in the path. The B-spline curve is:

[0097]

[0098] Where P i are the vertices of the characteristic polygon, which are given by path(P 0 ,P 1 ,...) is used to construct the point type of the spline, B i,k is the k-order (k-1) basis function, and after substituting it into the path, we get:

[0099]

[0100] B i,1 (u)=1,u i <u<u i+1 ; B i,k (u)=0,Otherwise.

[0101] Step 9: Set the UAV constraint function, introduce the performance constraints of the UAV's yaw angle and pitch angle, filter out some subsequent nodes, reduce the calculation complexity, and improve the fit and real-time performance of path planning;

[0102] Let the current node of the drone be X i , the next node selected by the drone is X i+1 , the previous parent node is X i-1 , its heading angle φ(X i ,X i+1 ) and pitch angle The formula is as follows:

[0103]

[0104] In the formula, (x i ,y i ,z i ) is node X i The three-dimensional coordinates of (x i+1 ,y i+1 ,z i+1 ) is the next node X i+1 The three-dimensional coordinates of (x i-1 ,y i-1 ,z i-1 ) is the previous parent node X i-1 The three-dimensional coordinates of

[0105]

[0106] The restriction set of the drone when selecting nodes is expressed as:

[0107]

[0108] Where N is point X i The set of all successor nodes of max and Indicates the maximum heading angle and pitch angle affected by the performance of the drone;

[0109] Step 10: The drone flies along the optimal path output by the improved HLOA. Every time the drone reaches a target point, it uses the onboard RGB-D camera to photograph the exterior wall of the high-rise building.

[0110] S3: When the drone flies along the planned path, every time it reaches a target point, it uses the onboard RGB-D camera to take pictures of the exterior wall defects of the high-rise building, and uses the novel MUME algorithm to identify and classify the exterior wall defects of the high-rise building;

[0111] A dataset of high-rise building exterior wall disease images is established based on the collected images. The high-rise building exterior wall diseases mainly include transverse cracks and longitudinal cracks. The present invention proposes to use a joint MUME algorithm to identify and classify high-rise building exterior wall disease phenomena. The identification process of the joint MUME algorithm is to use the MSRCR algorithm to enhance the collected high-rise building exterior wall disease images, obtain an accurate crack segmentation map through an improved U-Net model and perform binarization, and then use morphological filtering to denoise and smooth the binary crack image. Finally, a mathematical model of the EfficientNetv2-AttnConv network is constructed to classify and identify high-rise building disease cracks and cracks.

[0112] Since the image quality in the data set varies, the collected image data set needs to be processed. The purpose of preprocessing is to optimize data information, enhance the valuable features of the image, and improve the accuracy of recognition;

[0113] Step 1: Use the MSRCR algorithm to enhance the image. The multi-scale color enhancement (MSRCR) algorithm decomposes the image into multiple scales, enhances the image of each scale using the Retinex algorithm, obtains the reflectance and brightness components, and then performs color correction and weighted synthesis. The entire algorithm has low computational complexity and is relatively fast. When the MSRCR algorithm is enhanced by the Retinex algorithm, different parameters are used for images of different scales, which can achieve better enhancement effects. In addition, the color of the image can be kept unchanged during the enhancement process, and there will be no oversaturation or color cast problems due to enhancement. The formula of MSRCR is as follows:

[0114]

[0115] in:

[0116] C i (x,y)=β(log(αI i (x,y))-log(∑ i∈{r,g,b} I i (x,y)))

[0117] In the formula, β represents the gain value, and the size of α is related to the strength of nonlinearity; β=46 and α=125 are empirical parameters. MSRCR includes necessary color restoration, which can reflect the image details that cannot be seen by the naked eye in the dark area, and solve the problem of color distortion.

[0118] Step 2: Use the improved U-Net model to obtain an accurate crack segmentation map and perform binarization. Use the multi-scale feature prediction fusion neural network to improve the U-Net network. The improved U-Net network retains the original convolution kernel pooling part, and this idea is fused and predicted to act on the features of the middle layer of the U-Net network. This method can improve the network's receptive field, obtain contextual information in different situations, and enhance the network's ability to understand and judge target features. The output formula of the multi-scale feature prediction fusion is expressed as:

[0119] Z(x)=Conv([Z 0 ,Z 1 ,Z 2 ,Z 3 ,Z 4 ])

[0120] In the formula, Z(x) represents the output after feature prediction fusion, Z 0 ,Z 1 ,Z 2 ,Z 3 ,Z 4 Represented as upsampled output feature maps of different scales;

[0121] The loss function design has a great impact on the performance of the network model. In the past, most image segmentation tasks used cross entropy as the target loss function. The cross entropy loss function is used for pixel-level classification. For crack images, the target cracks account for a small proportion, which will lead to the problem of class imbalance. Therefore, the Dice loss function is introduced.

[0122] Considering the advantages and disadvantages of the cross entropy loss function and the Dice loss function, the two loss functions are weighted and summed to obtain a new loss function; the new loss function formula is:

[0123]

[0124] L=αL CE +(1-α)L Dice

[0125] Where N is the number of pixels, y is i represents the label of the i-th pixel, Represents the prediction result of the i-th pixel; ε is a hyperparameter to prevent the denominator from being 0; α∈[0,1] is the weight coefficient;

[0126] Finally, the obtained crack segmentation map is binarized;

[0127] Step 3: Use morphological filtering to denoise and smooth the crack binary image. Morphological filtering is a nonlinear filtering technology. Its core idea is to construct specific structural elements to perform front-to-back translation matching or local correction on the original signal, thereby suppressing noise while retaining the morphological characteristics of the signal. It can effectively eliminate some isolated small points or smaller connected areas through operations such as corrosion and expansion, and can also eliminate burrs and uneven parts, so as to achieve the purpose of smoothing the image edge.

[0128] The present invention selects the opening and closing operation in morphological processing to filter the binary image; the opening and closing operation is mainly based on the combination of corrosion and expansion operators, and the formula is:

[0129]

[0130] In the formula, A represents the object to be processed; B is the structural element; is the expansion symbol; Θ is the erosion symbol;

[0131] In order to obtain rich edge detail information and enhance the operator's ability to resist noise interference, an improved noise-resistant morphological operator is used, and the formula is:

[0132]

[0133] Where ε is the detail factor; E max =max{E d ,E e},E min =min{E d ,E e}; i is the Mahalanobis distance weighting coefficient; b 1 is a 2×2 square structural element; b 2i Represents 3×3 linear structural elements in four different directions of 0°, 45°, 90° and 135°;

[0134] Step 4: Divide the high-rise building disease crack images into a test set and a training set and input them into the EfficientNetv2-A model. Train the mathematical model of the EfficientNetv2-A network to classify and identify high-rise building disease cracks and fissures. EfficientNetv2-A introduces a convolution operator AttnConv based on the EfficientNetv2 model. In AttnConv, firstly, Q, K, and V are obtained through linear transformation. The formula is:

[0135] Q,K,V=FC(X in )

[0136] In the formula, X in is the input of AttnConv, FC stands for fully connected layer;

[0137] After completing the linear transformation, deep convolution (DWconv) is used on V to aggregate local features with shared weights. The formula is:

[0138] V S =DWconv(V)

[0139] Next, we implement context-aware local enhancement, the formula is:

[0140] Q l =DWconv(Q)

[0141] K l =DWconv(K)

[0142] Attn t =FC(Swish(FC(Q l ⊙K l )))

[0143]

[0144] X local =Attn·V S

[0145] Where d is the number of channels, ⊙ represents the Hardmard product;

[0146] Finally, a fully connected layer is applied on the channel dimension:

[0147] X t =Concat(X local ,X global )

[0148] X out =FC(X t )

[0149] The network model loss function of the present invention is based on the improved weighted cross entropy loss and combined with the Dice loss function, and the formula is:

[0150]

[0151] In the formula, ε is a minimum value to prevent the denominator from being 0; the Dice loss function of the side network is:

[0152] The total side loss function formula is:

[0153]

[0154] Connect the side output layers to get the final fusion layer. The loss formula of the fusion layer is:

[0155] L fuse (X,Y,W)=L′ WCE (X,Y,W)+L Dice (X,Y,W)

[0156] The overall loss function formula for network model training is:

[0157] L=L side (X,Y,W,w)+L fuse (X,Y,W).

Claims

1. The operation method of the high-rise building exterior wall disease identification system based on drone is characterized by: The RGB-D camera on the drone is used to complete the flight path planning, and then the RGB-D camera is used to complete the identification and classification of the external wall disease phenomena of the high-rise building. The operation steps are as follows: Step (1): The drone uses the laser radar and RGB-D camera to scan and model the high-rise environment, obtain point cloud data, and set target points; Step (2): In the process of UAV path planning, the improved HLOA is used for path planning, in which the UAV trajectory is constrained by using the path smoothing method and constraint function to obtain the shortest and safest UAV flight trajectory; Step (3): When the drone flies along the planned path, every time it reaches a target point, it uses the onboard RGB-D camera to photograph the exterior wall defects of the high-rise building, and uses the novel MUME algorithm to identify and classify the exterior wall defects of the high-rise building.

2. The operating method of the high-rise building exterior wall disease identification system based on drone according to claim 1 is characterized in that: In step (1), the specific method of collecting high-rise building environment information by the laser radar and RGB-D camera is: According to the laser radar and RGB-D camera, define the m target point set for acquisition: T = t1, t2, ..., t m and the initial position of the drone: S = (x0, y0, z0); Define the path planning sequence of the drone based on the lidar: P = p1, p2, ..., p m+1 , In the formula, p i Indicates the location of the drone at step i.

3. The operating method of the high-rise building exterior wall disease identification system based on drone according to claim 1 is characterized in that: In step (2), the specific method of using the improved HLOA for path planning is: First, generate the initial population defined as is the i-th horned lizard in the t-th generation. The Logistic map of the chaotic map is introduced, and its randomness and ergodicity are used to improve the search ability. At the same time, the fitness of each horned lizard is calculated; The mathematical expression of Logistic mapping is expressed as: Z i (t)=μZ i (t-1)(1-Z i (t-1)) Where t is the iteration time, x(t)∈[0,1], μ is an adjustable parameter, μ∈[0,4]; Z(t) is the proportion of the population at time t to the maximum possible population size; The horned lizard individuals added to the logistic map are updated by the following formula: Make the initial population evenly distributed in the space and make the obtained initial population closer to the optimal solution; When a horned lizard detects danger, it will choose different strategies according to the situation. First, it will create a hidden strategy and define is the i-th search agent in the t+1-th generation space, is the search agent with the best fitness in the tth generation, and the calculation formula is: Where r1, r2, r3 and r4 are integer random numbers generated between 1 and the maximum number of search agents, and satisfy r1≠r2≠r3≠r4; Max_iter represents the maximum number of iterations; σ is a binary number that determines whether the random number from the color table is less than or equal to 0.

5. If it is satisfied, it is 1, otherwise it is 0; Set to 2, c1 and c2 are simulated chromaticities and c1≠c2.

4. The operating method of the high-rise building exterior wall disease identification system based on drone according to claim 3 is characterized in that: The improved horned lizard algorithm abandons the attack strategy and chooses to create a mobile escape strategy to improve the drone's handling of obstacles. The formula is: Where walk is a random number generated between -1 and 1; ε is a random number generated by a standard Cauchy distribution, where σ is set to 0 and 1; The worst search agent is replaced by skin lightening and darkening strategies, the formula is: In the formula, x worst (t) is the search agent with the worst fitness in the tth generation; Light1 and Light2 are random numbers generated between 0 and 0.4046661; Dark1 and Dark2 are random numbers generated between 0.5440510 and 1.

5. The operating method of the high-rise building exterior wall disease identification system based on drone according to claim 4 is characterized in that: In order to avoid the algorithm from falling into the problem of premature convergence and jumping out of the local optimum, a random difference mutation strategy is introduced to use the position of the current best individual and the position of the randomly selected individual to perform random difference processing to generate a new position; the specific formula is as follows: In the formula, r5 represents a random number in the range [0,1]; represents the latest individual in generation t+1; represents a random individual in the tth generation; The planned path is smoothed and the B-spline algorithm is used to process the redundant transformations and corners in the path. The B-spline curve is: Where P i are the vertices of the characteristic polygon, which are used by path(P0,P1,...) to construct the point type of the spline. i,k is the k-order (k-1) basis function, and after substituting it into the path, we get: B i,1 (u)=1,u i <u<u i+1 ;B i,k (u)=0,Otherwise. Set the drone constraint function, introduce the performance constraints of the drone's yaw angle and pitch angle, and filter out some subsequent nodes; set the drone's current node as X i , the next node selected by the drone is X i+1 , the previous parent node is X i-1 , its heading angle φ(X i ,X i+1 ) and pitch angle The formula is as follows: In the formula, (x i ,y i ,z i ) is node X i The three-dimensional coordinates of (x i+1 ,y i+1 ,z i+1 ) is the next node X i+1 The three-dimensional coordinates of (x i-1 ,y i-1 ,z i-1 ) is the previous parent node X i-1 The three-dimensional coordinates of The restriction set of the drone when selecting nodes is expressed as: Where N is point X i The set of all successor nodes of max and Indicates the maximum heading and pitch angles affected by the drone's performance.

6. The operating method of the high-rise building exterior wall disease identification system based on drone according to claim 1 is characterized in that: In step (3), the specific method of using the novel MUME algorithm to identify and classify high-rise building exterior wall disease phenomena is: The drone improves the optimal path flight output by HLOA. Whenever the drone reaches a target point, it uses the onboard RGB-D camera to photograph the exterior wall of the high-rise building. Based on the collected images, a dataset of image defects on the exterior wall of the high-rise building is established. The defects on the exterior wall of the high-rise building include horizontal cracks and vertical cracks. The joint MUME algorithm is used to identify and classify the defects of high-rise building exterior walls. The recognition process of the joint MUME algorithm is to perform image enhancement on the collected images of high-rise building exterior wall defects using the MSRCR algorithm, obtain accurate crack segmentation maps through the improved U-Net model and perform binarization, and then use morphological filtering to denoise and smooth the binary image of the cracks. Finally, the mathematical model of the EfficientNetv2-AttnConv network is constructed to classify and identify cracks and fissures in high-rise building defects.

7. The operating method of the high-rise building exterior wall disease identification system based on drone according to claim 6 is characterized in that: The MSRCR algorithm is used for image enhancement and multi-scale color enhancement. The MSRCR algorithm decomposes the image into multiple scales, and enhances the image of each scale using the Retinex algorithm to obtain the reflectivity and brightness components, and then performs color correction and weighted synthesis. When the MSRCR algorithm is enhanced by the Retinex algorithm, different parameters are used for images of different scales, and the color of the image can be kept unchanged during the enhancement process, without causing oversaturation or color cast problems due to enhancement; the formula of MSRCR is as follows: C i (x,y)=β(log(αI i (x,y))-log(∑ i∈{r,g,b} I i (x,y))) In the formula, β represents the gain value, and the size of α is related to the strength of nonlinearity.

8. The operating method of the high-rise building exterior wall disease identification system based on drone according to claim 7 is characterized in that: The crack segmentation map is obtained and binarized using the improved U-Net model. The multi-scale feature prediction fusion neural network is used to improve the U-Net network. The improved U-Net network retains the original convolution kernel pooling part and applies this fusion prediction to the features of the middle layer of the U-Net network. It can obtain contextual information in different situations and enhance the network's ability to understand and judge target features; The output formula of the multi-scale feature prediction fusion is expressed as: Z(x)=Conv([Z0,Z1,Z2,Z3,Z4]) Where Z(x) represents the output after feature prediction fusion, and Z0, Z1, Z2, Z3, and Z4 represent upsampled output feature maps of different scales.

9. The operating method of the high-rise building exterior wall disease identification system based on drone according to claim 8 is characterized in that: The loss function design has an impact on the performance of the network model. In the past, most image segmentation tasks used cross entropy as the target loss function. The cross entropy loss function is used for pixel-level classification. For crack images, the target cracks account for a small proportion, so the Dice loss function is introduced. Considering the advantages and disadvantages of the cross entropy loss function and the Dice loss function, the two loss functions are weighted and summed to obtain a new loss function; the new loss function formula is: L=αL CE +(1-α)L Dice Where N is the number of pixels, y is i represents the label of the i-th pixel, Represents the prediction result of the i-th pixel; ε is a hyperparameter to prevent the denominator from being 0; α∈[0,1] is the weight coefficient; Finally, the obtained crack segmentation map is binarized.

10. The operating method of the high-rise building exterior wall disease identification system based on drone according to claim 9 is characterized in that: Morphological filtering is used to denoise and smooth the crack binary image. Morphological filtering constructs specific structural elements to perform front-to-back translation matching or local correction on the original signal, thereby suppressing noise while retaining the morphological characteristics of the signal. It eliminates isolated small spots or smaller connected areas by corrosion expansion, while also eliminating burrs and uneven parts; The opening and closing operations in morphological processing are selected to filter the binary image; the opening and closing operations are mainly based on the combination of corrosion and expansion operators, and the formula is: In the formula, A represents the object to be processed; B is the structural element; is the expansion symbol; Θ is the erosion symbol; The images of high-rise building defects and cracks are divided into a test set and a training set and input into the EfficientNetv2-A model. The mathematical model of the EfficientNetv2-A network is trained to classify and identify high-rise building defects and cracks.

Images