Method for operating a drone-based system for identifying defects on the outer wall of a high-rise building

Abstract

The application discloses a kind of based on high-rise outer wall disease identification system of unmanned aerial vehicle operation method;Belong to image detection technical field, its implementation steps are as follows: through unmanned aerial vehicle carries RGB-D camera to complete flight path planning, while photographing identification classification to the disease of high-rise outer wall;The flight path planning process of unmanned aerial vehicle is that unmanned aerial vehicle carries RGB-D camera to complete the establishment of high-rise environment model, the flight path of unmanned aerial vehicle is planned using improved HLOA, to obtain the shortest, safest unmanned aerial vehicle flight path;Camera is carried in the process of unmanned aerial vehicle flight path, and high-rise outer wall disease information is collected, and a novel algorithm MUME proposed in the application is used to classify and process high-rise outer wall crack and crack disease image information, and the novel MUME algorithm proposed in the application can more quickly and accurately identify and detect high-rise outer wall disease information.

Description

Technical Field

[0001] This invention belongs to the field of image detection technology and relates to an operation method of a high-rise building exterior wall defect identification system based on drones. Background Technology

[0002] High-rise buildings are an important infrastructure for people's lives. Their emergence has not only changed people's lives but also brought a series of challenges and problems. Due to long-term use and natural disasters, various defects such as cracks and fissures may appear on the surface of high-rise buildings. If these defects are not repaired, maintained and managed in a timely manner, they will seriously affect the safety and stability of high-rise buildings and may even cause disasters.

[0003] Currently, the identification and detection of surface defects in high-rise buildings mainly rely on manual inspection and visual inspection. However, manual inspection is limited by manpower, resources, and time, while visual inspection also has certain limitations in accuracy and reliability. These factors hinder the discovery and treatment of surface defects in high-rise buildings. Therefore, developing an efficient, accurate, reliable, and intelligent method for identifying surface defects in high-rise buildings is crucial for ensuring the safety and stability of high-rise buildings and reducing the occurrence of safety accidents. Summary of the Invention

[0004] To address the aforementioned problems, the present invention aims to propose an operation method for a high-rise building exterior wall defect identification system based on unmanned aerial vehicles (UAVs), with the goal of improving the identification accuracy of high-rise building defects.

[0005] The technical solution of this invention is as follows: The operation method of the high-rise building exterior wall defect identification system based on UAV described in this invention involves using a UAV equipped with an RGB-D camera to complete flight path planning, and then using the equipped RGB-D camera to complete the identification and classification of high-rise building exterior wall defects. The operation steps are as follows:

[0006] Step (1): The UAV uses its onboard LiDAR and RGB-D camera to scan and model the high-rise building environment, acquire point cloud data, and set target points;

[0007] Step (2): In the process of UAV path planning, the improved horned lizard algorithm (HLOA) is used for path planning, in which path smoothing method and constraint function are used to constrain the UAV trajectory to obtain the shortest and safest UAV flight trajectory;

[0008] Step (3): When the UAV flies along the planned path, it takes pictures of the defects on the exterior wall of the high-rise building with the onboard RGB-D camera whenever the UAV reaches a target point, and uses the novel MUME algorithm to identify and classify the defects on the exterior wall of the high-rise building.

[0009] Furthermore, in step (1), the specific method for the lidar and RGB-D camera to collect high-rise building environmental information is as follows:

[0010] Based on the definition and configuration of the LiDAR and RGB-D camera, a set of m target points is collected: T = t1, t2, ..., t m The initial position of the drone: S = (x0, y0, z0);

[0011] Based on the lidar definition, the path planning sequence for the UAV is: P = p1, p2, ..., p m+1 In the formula, p i This indicates the location of the drone at step i.

[0012] Furthermore, in step (2), the improved HLO algorithm is used for path planning, and the specific method is as follows:

[0013] First, an initial population (search agent) is generated, defined as follows: It is the i-th horned lizard in the t-th generation (time); the Logistic mapping with chaotic mapping is introduced, and its randomness and ergodicity are used to improve the search ability, while the fitness of each horned lizard is calculated.

[0014] The mathematical expression for the Logistic mapping is:

[0015] Z i (t)=μZ i (t-1)(1-Z i (t-1))

[0016] In the formula, t is the iteration time, x(t)∈[0,1], μ is an adjustable parameter, μ∈[0,4]; Z(t) is the proportion of the population to the maximum possible population size at time t (i.e., the ratio of the existing population size to the maximum possible population size). In this method, μ is taken as 4.

[0017] Individual horned lizards added to the Logistic mapping are updated using the following formula:

[0018]

[0019] The goal is to ensure that the initial population is evenly distributed in space and that the obtained initial population is closer to the optimal solution.

[0020] When the horned lizard detects danger, it will choose different strategies depending on the situation, first establishing a "stealth strategy"; defining... It is the i-th search agent in the (t+1)-th generation space. It is the search agent with the best fitness in generation t, and the calculation formula is:

[0021]

[0022] In the formula, r1, r2, r3, and r4 are random integers generated between 1 and the maximum number of search agents, and satisfy r1≠r2≠r3≠r4; Max_iter represents the maximum number of iterations; σ is a binary number that is 1 if the random number from the color table is less than or equal to 0.5, and 0 otherwise. Set to 2, c1 and c2 are analog chromaticities and c1≠c2;

[0023] The improved HLOA abandons the "attack strategy" which has low path planning efficiency, and instead chooses to create a "mobile escape strategy" to improve the drone's obstacle handling and enhance safety. The formula is:

[0024]

[0025] In the formula, walk is a random number generated between -1 and 1; ε is a random number generated by the standard Cauchy distribution, where σ is set to 0 and 1;

[0026] Replace the worst search agent with a "skin brightening and darkening strategy", the formula is:

[0027]

[0028] In the formula, It is the search agent with the worst fitness in generation t; Light1 and Light2 are random numbers generated between 0 and 0.4046661; Dark1 and Dark2 are random numbers generated between 0.5440510 and 1;

[0029] To avoid premature convergence and escaping local optima, a random difference mutation strategy is introduced. This strategy uses the position of the current best individual and the position of a randomly selected individual to perform random difference processing to generate a new position. The specific formula is as follows:

[0030]

[0031] In the formula, r5 represents a random number in the range [0,1]. This represents the latest individual in generation t+1; Represents a random individual in generation t;

[0032] The planned path is smoothed using the B-spline algorithm to handle redundant transformations and corners. The B-spline curve is as follows:

[0033]

[0034] In the formula, P iThese are the vertices of the feature polygon, which are the points used by path(P0,P1,...) to construct the spline. B i,k Let the basis function be of order k (k-1), then substituting it into path yields:

[0035]

[0036] B i,1 (u)=1,u i <u<u i+1 B i,k (u) = 0, Otherwise.

[0037] Set up a constraint function for the UAV, introduce performance constraints on the UAV's yaw and pitch angles, filter out some successor nodes, reduce computational complexity, and improve the efficiency and real-time performance of path planning.

[0038] Let the current node of the drone be X. i The next node selected by the drone is X. i+1 The previous parent node is X i-1 Its deviation angle φ(X) i ,X i+1 ) and pitch angle The formula is as follows:

[0039]

[0040] In the formula, (x i ,y i ,z i ) is node X i The three-dimensional coordinates; (x i+1 ,y i+1 ,z i+1 ) is the next node X i+1 The three-dimensional coordinates; (x i-1 ,y i-1 ,z i-1 ) is the previous parent node X i-1 3D coordinates;

[0041]

[0042] The set of constraints for drones when selecting nodes is represented as follows:

[0043]

[0044] In the formula, N is the point X. i The set of all successor nodes, φ max and Indicates the maximum heading and pitch angles affected by the performance of the drone.

[0045] Furthermore, in step (3), the specific method for identifying and classifying high-rise building exterior wall defects using the novel MUME algorithm is as follows:

[0046] The drone flies along the optimal path for improving HLOA output. Whenever the drone reaches a target point, it uses an onboard RGB-D camera to take pictures of the exterior wall of the high-rise building. Based on the collected images, a dataset of images of defects on the exterior wall of the high-rise building is established. The defects on the exterior wall of the high-rise building mainly include transverse cracks and longitudinal cracks.

[0047] The joint MUME algorithm is used to identify and classify defects on the exterior walls of high-rise buildings. The identification process of the joint MUME algorithm is as follows: the MSRCR algorithm is used to enhance the collected images of defects on the exterior walls of high-rise buildings; an accurate crack segmentation map is obtained through the improved U-Net model and binarized; then morphological filtering is used to denoise and smooth the binarized crack image; finally, the mathematical model of EfficientNetv2-AttnConv network is constructed to classify and identify cracks and fissures in high-rise buildings.

[0048] The MSRCR algorithm is used for image enhancement, specifically multi-scale color enhancement. The MSRCR algorithm decomposes the image into multiple scales, applies Retinex enhancement to each scale to obtain reflectance and luminance components, and then performs color correction and weighted synthesis. The overall algorithm has low computational complexity and is relatively fast. During Retinex enhancement, the MSRCR algorithm uses different parameters for different scales, resulting in better enhancement effects. Furthermore, it maintains color integrity during enhancement, avoiding oversaturation or color cast. The MSRCR formula is as follows:

[0049]

[0050] C i (x,y)=β(log(αI i (x,y))-log(∑ i∈{r,g,b} I i (x,y)))

[0051] In the formula, β represents the gain value, and the magnitude of α is related to the intensity of nonlinearity;

[0052] An improved U-Net model is used to obtain accurate crack segmentation maps, which are then binarized. A multi-scale feature prediction fusion neural network is applied to improve the U-Net network. The improved U-Net network retains the original convolutional kernel pooling parts, and this fused prediction is applied to the features of the intermediate layers of the U-Net network. This approach can increase the network's receptive field, obtain contextual information under different conditions, and enhance the network's understanding and judgment ability of target features. The output formula of this multi-scale feature prediction fusion is expressed as:

[0053] Z(x) = Conv([Z0,Z1,Z2,Z3,Z4])

[0054] In the formula, Z(x) represents the output after feature prediction fusion, and Z0, Z1, Z2, Z3, Z4 represent upsampled output feature maps at different scales.

[0055] The design of the loss function has a significant impact on the performance of the network model. Previously, most image segmentation tasks used cross-entropy as the target loss function. While cross-entropy loss is used for pixel-level classification, it can lead to class imbalance in crack images where the target crack represents a small proportion. Therefore, the Dice loss function is introduced. Considering the advantages and disadvantages of both cross-entropy and Dice loss functions, a weighted sum of the two loss functions is obtained, resulting in a new loss function. The formula for the new loss function is:

[0056]

[0057] L=αL CE +(1-α)L Dice

[0058] In the formula, N is the number of pixels, and y i This represents the label of the i-th pixel. This represents the prediction result for the i-th pixel; ε is a hyperparameter to prevent the denominator from being 0; α∈[0,1] is the weight coefficient;

[0059] Finally, the obtained crack segmentation map is binarized.

[0060] Morphological filtering is used to denoise and smooth the image after the crack is binarized. Morphological filtering is a nonlinear filtering technique. Its core idea is to construct specific structuring elements to perform front-to-back translation matching or local correction on the original signal, suppressing noise while preserving the morphological features of the signal. Through operations such as erosion and dilation, it can effectively eliminate some isolated small points or small connected regions, and at the same time eliminate burrs and uneven parts, so as to smooth the image edges.

[0061] This invention selects opening and closing operations from morphological processing for filtering binarized images; the opening and closing operation is mainly based on a combination of erosion and dilation operators, and the formula is:

[0062]

[0063] In the formula, A represents the object to be processed; B is a structural element; Θ is the expansion symbol; θ is the erosion symbol.

[0064] Images of high-rise building defects and cracks are divided into test and training sets and input into the EfficientNetv2-A model. The mathematical model of the EfficientNetv2-A network is trained to classify and identify high-rise building defects, cracks, and fissures.

[0065] The beneficial effects of this invention are: 1. Considering the actual environment, UAVs need to constantly adapt to the high-rise building environment; therefore, the optimal path will be relatively complex during the UAV path planning process. Considering the real environment, this invention adopts an improved horn lizard algorithm that integrates path smoothing and constraint functions to constrain the UAV trajectory, so as to improve the safety and rationality of UAV flight; 2. This invention proposes to use a novel MUME algorithm to identify and classify information on defects in the exterior walls of high-rise buildings. The process of the novel MUME algorithm is as follows: the MSRCR algorithm performs image enhancement on the collected images of defects in the exterior walls of high-rise buildings, obtains accurate crack segmentation maps through the improved U-Net model and performs binarization, then uses morphological filtering to denoise and smooth the binarized crack images, and finally constructs a mathematical model of EfficientNetv2-AttnConv network to classify and identify cracks and fissures in high-rise buildings. Attached Figure Description

[0066] Figure 1 This is a flowchart of an embodiment of the present invention;

[0067] Figure 2 This is a flowchart of the novel MUME algorithm in an embodiment of the present invention. Detailed Implementation

[0068] The specific technical solution of the present invention will be further described in detail below with reference to specific examples.

[0069] As shown in the figure, the operation method of the high-rise building exterior wall defect identification system based on UAV described in this invention involves scanning and modeling the high-rise building environment using a lidar and RGB-D camera mounted on the UAV, completing path planning using an improved horned lizard algorithm, and then identifying and classifying the high-rise building exterior wall defects using the mounted RGB-D camera. The specific implementation process is as follows:

[0070] S1: The drone uses its onboard LiDAR and RGB-D camera to scan and model the high-rise building environment, acquire point cloud data, and set target points;

[0071] Specifically, the drone is first started, the model aircraft battery begins to power the various modules of the drone, the motor drives the drone wings to start working, and the drone is driven to reach any coordinate position in the space by controlling the four wings. Then the lidar and RGB-D camera on the drone are activated to scan and model the high-rise environment, thereby obtaining the approximate shape of the high-rise, and point cloud data is obtained through the depth sensor on the camera.

[0072] The target point is set based on the collected point cloud data, and the improved HLOA is used to complete the path planning of the UAV. When the UAV moves through all viewpoint coordinate positions, the high-rise building inspection task is completed.

[0073] S2: In the process of UAV path planning, the improved lizard algorithm (HLOA) is used for path planning, which uses path smoothing methods and constraint functions to constrain the UAV trajectory in order to obtain the shortest and safest UAV flight trajectory.

[0074] Specifically, considering the real-world environment, drones need to constantly adapt to the complex environment of high-rise buildings; therefore, the optimal path in the drone trajectory planning process will be quite complex and contain many different features. Therefore, the improved HLOA algorithm is used for drone path planning.

[0075] Step 1: Define the set of m target points for data acquisition: T = t1, t2, ..., t m Initial position of the UAV: ​​S = (x0, y0); Define the distance matrix between the UAV and the target point: D ij =d ij (m+1×m+1), where d ij This represents the Euclidean distance from point i to point j;

[0076] Step 2: Define the path planning sequence for the UAV: ​​P = p1, p2, ..., p m+1 In the formula, p i This indicates the location of the drone at step i;

[0077] Step 3: Input the defined points and high-rise building modeling information into the improved horned lizard algorithm; generate the initial population (search agent) defined as follows. It is the i-th horned lizard in the t-th generation (time); the Logistic mapping with chaotic mapping is introduced, and its randomness and ergodicity are used to improve the search ability, while the fitness of each horned lizard is calculated.

[0078] The mathematical expression for the Logistic mapping is:

[0079] Z i (t)=μZ i (t-1)(1-Z i (t-1))

[0080] In the formula, t is the iteration time, x(t)∈[0,1], μ is an adjustable parameter, μ∈[0,4]; Z(t) is the proportion of the population to the maximum possible population size at time t (i.e., the ratio of the existing population size to the maximum possible population size), and μ is taken as 4 in this invention;

[0081] Individual horned lizards added to the Logistic mapping are updated using the following formula:

[0082]

[0083] The goal is to ensure that the initial population is evenly distributed in space and that the obtained initial population is closer to the optimal solution.

[0084] Step 4: When the horned lizard detects danger, it will choose different strategies depending on the situation. First, it establishes a "stealth strategy"; define... It is the i-th search agent in the (t+1)-th generation space. It is the search agent with the best fitness in generation t, and the calculation formula is:

[0085]

[0086] In the formula, r1, r2, r3, and r4 are random integers generated between 1 and the maximum number of search agents, and satisfy r1≠r2≠r3≠r4; Max_iter represents the maximum number of iterations; σ is a binary number that is 1 if the random number from the color table is less than or equal to 0.5, and 0 otherwise. Set to 2, c1 and c2 are analog chromaticities and c1≠c2;

[0087] Step 5: The improved HLOA abandons the "attack strategy" which has low path planning efficiency and instead chooses to create a "mobile escape strategy" to improve the drone's obstacle handling and enhance safety. The formula is:

[0088]

[0089] In the formula, walk is a random number generated between -1 and 1; ε is a random number generated by the standard Cauchy distribution, where σ is set to 0 and 1;

[0090] Step 6: Replace the worst search proxy with a "skin brightening and darkening strategy", the formula is:

[0091]

[0092] In the formula, It is the search agent with the worst fitness in generation t; Light1 and Light2 are random numbers generated between 0 and 0.4046661; Dark1 and Dark2 are random numbers generated between 0.5440510 and 1;

[0093] Step 7: To avoid premature convergence and escaping local optima, a random difference mutation strategy is introduced. This involves randomly differencing the current best individual's position with the position of a randomly selected individual to generate a new position. The specific formula is as follows:

[0094]

[0095] In the formula, r5 represents a random number in the range [0,1]. This represents the latest individual in generation t+1; Represents a random individual in generation t;

[0096] Step 8: Smooth the planned path using the B-spline algorithm to handle redundant transformations and corners. The B-spline curve is as follows:

[0097]

[0098] In the formula, P i These are the vertices of the feature polygon, which are the points used by path(P0,P1,...) to construct the spline. B i,k Let the basis function be of order k (k-1), then substituting it into path yields:

[0099]

[0100] B i,1 (u)=1,u i <u<u i+1 B i,k (u) = 0, Otherwise.

[0101] Step 9: Set up UAV constraint functions, introduce performance constraints on UAV yaw and pitch angles, filter out some successor nodes, reduce computational complexity, and improve the efficiency and real-time performance of path planning.

[0102] Let the current node of the drone be X. i The next node selected by the drone is X. i+1 The previous parent node is X i-1 Its deviation angle φ(X) i ,X i+1 ) and pitch angle The formula is as follows:

[0103]

[0104] In the formula, (x i ,y i ,z i ) is node X i The three-dimensional coordinates; (x i+1,y i+1 ,z i+1 ) is the next node X i+1 The three-dimensional coordinates; (x i-1 ,y i-1 ,z i-1 ) is the previous parent node X i-1 3D coordinates;

[0105]

[0106] The set of constraints for drones when selecting nodes is represented as follows:

[0107]

[0108] In the formula, N is the point X. i The set of all successor nodes, φ max and Indicates the maximum heading and pitch angles affected by the performance of the drone;

[0109] Step 10: The drone flies along the optimal path of the improved HLOA output. Whenever the drone reaches a target point, it uses the onboard RGB-D camera to take pictures of the exterior wall of the high-rise building.

[0110] S3: When the drone flies along the planned path, it uses the onboard RGB-D camera to take pictures of the defects on the exterior walls of the high-rise building whenever it reaches a target point. The novel MUME algorithm is used to identify and classify the defects on the exterior walls of the high-rise building.

[0111] A dataset of high-rise building exterior wall defects is established based on the collected images. High-rise building exterior wall defects mainly include horizontal cracks and vertical cracks. This invention proposes to use the joint MUME algorithm to identify and classify high-rise building exterior wall defects. The identification process of the joint MUME algorithm is as follows: the MSRCR algorithm is used to enhance the collected high-rise building exterior wall defect images; an accurate crack segmentation map is obtained through the improved U-Net model and binarized; then morphological filtering is used to denoise and smooth the binarized crack image; finally, the mathematical model of EfficientNetv2-AttnConv network is constructed to classify and identify high-rise building defects such as cracks and fissures.

[0112] Because the image quality varies in the dataset, the acquired image dataset needs to be processed. The purpose of preprocessing is to optimize the data information, enhance the valuable features of the images, and improve the accuracy of recognition.

[0113] Step 1: Image enhancement using the MSRCR algorithm. The Multi-Scale Color Enhancement (MSRCR) algorithm decomposes the image into multiple scales, applies Retinex enhancement to each scale, and obtains reflectance and luminance components. Then, color correction and weighted synthesis are performed. The overall algorithm has low computational complexity and is relatively fast. When using Retinex enhancement, the MSRCR algorithm uses different parameters for different scales, resulting in better enhancement effects. Furthermore, it maintains the image's color during enhancement, avoiding oversaturation or color cast issues. The MSRCR formula is as follows:

[0114]

[0115] in:

[0116] C i (x,y)=β(log(αI i (x,y))-log(∑ i∈{r,g,b} I i (x,y)))

[0117] In the formula, β represents the gain value, and the magnitude of α is related to the intensity of nonlinearity; where β = 46 and α = 125 are empirical parameters. MSRCR contains necessary color restoration, which can reveal image details that are invisible to the naked eye in dark areas, while solving the problem of color distortion.

[0118] Step 2: Obtain accurate crack segmentation maps using the improved U-Net model and perform binarization; use a multi-scale feature prediction fusion neural network to improve the U-Net network. The improved U-Net network retains the original convolutional kernel pooling part, and this idea is fused to predict features applied to the intermediate layers of the U-Net network; this method can increase the network's receptive field, obtain contextual information under different conditions, and enhance the network's understanding and judgment ability of target features; the output formula of this multi-scale feature prediction fusion is expressed as:

[0119] Z(x) = Conv([Z0,Z1,Z2,Z3,Z4])

[0120] In the formula, Z(x) represents the output after feature prediction fusion, and Z0, Z1, Z2, Z3, Z4 represent upsampled output feature maps at different scales;

[0121] The design of the loss function has a significant impact on the performance of the network model. In the past, most image segmentation tasks used cross-entropy as the target loss function. The cross-entropy loss function is used for pixel-level classification. For crack images, the target crack accounts for a small proportion, which can lead to class imbalance. Therefore, the Dice loss function is introduced.

[0122] Considering the advantages and disadvantages of the cross-entropy loss function and the Dice loss function, a weighted sum of the two loss functions is obtained to obtain a new loss function; the formula for the new loss function is:

[0123]

[0124] L=αL CE +(1-α)L Dice

[0125] In the formula, N is the number of pixels, and y i This represents the label of the i-th pixel. This represents the prediction result for the i-th pixel; ε is a hyperparameter to prevent the denominator from being 0; α∈[0,1] is the weight coefficient;

[0126] Finally, the obtained crack segmentation map is binarized;

[0127] Step 3: Morphological filtering is used to denoise and smooth the image after the crack binarization. Morphological filtering is a nonlinear filtering technique. Its core idea is to construct specific structuring elements to perform front-to-back translation matching or local correction on the original signal, suppressing noise while preserving the morphological features of the signal. Through operations such as erosion and dilation, it can effectively eliminate some isolated small points or small connected regions, and at the same time eliminate burrs and uneven parts, so as to achieve the purpose of smoothing the image edges.

[0128] This invention selects opening and closing operations from morphological processing for filtering binarized images; the opening and closing operation is mainly based on a combination of erosion and dilation operators, and the formula is:

[0129]

[0130] In the formula, A represents the object to be processed; B is a structural element; Θ is the expansion symbol; θ is the erosion symbol.

[0131] To capture rich detail at the edges and enhance the operator's resistance to noise, an improved noise-resistant morphological operator is employed, with the following formula:

[0132]

[0133] In the formula, ε is the detail factor; E max =max{E d E e},E min =min{E d E e};λ iThese are Mahalanobis distance weighting coefficients; b1 is a 2×2 square structuring element; b 2i Represents a 3×3 linear structure element with four different directions: 0°, 45°, 90°, and 135°.

[0134] Step 4: Divide the images of high-rise building defects and cracks into test and training sets and input them into the EfficientNetv2-A model. Train the mathematical model of the EfficientNetv2-A network to classify and identify high-rise building defects, cracks, and fissures. EfficientNetv2-A introduces a convolution operator AttnConv on the basis of the EfficientNetv2 model. In AttnConv, Q, K, and V are first obtained through linear transformation, with the following formula:

[0135] Q,K,V=FC(X in )

[0136] In the formula, X in It is the input of AttnConv, and FC represents a fully connected layer;

[0137] After the linear transformation, depthwise convolution (DWconv) is applied to V to aggregate local features with shared weights, as shown in the formula:

[0138] V S =DWconv(V)

[0139] Next, we implement context-aware local enhancement, using the following formula:

[0140] Q l =DWconv(Q)

[0141] K l =DWconv(K)

[0142] Attn t =FC(Swish(FC(Q) l ⊙K l )))

[0143]

[0144] X local =Attn·V S

[0145] In the formula, d is the number of channels, and ⊙ represents the Hardmard product;

[0146] Finally, a fully connected layer is applied at the channel dimension:

[0147] X t =Concat(X) local ,Xglobal )

[0148] X out =FC(X) t )

[0149] The network model loss function of this invention combines the improved weighted cross-entropy loss function with the Dice loss function, and the formula is as follows:

[0150]

[0151] In the formula, ε is a local minimum to prevent the denominator from being zero; the Dice loss function for the side network is:

[0152] The formula for the total side loss function is:

[0153]

[0154] By connecting the side output layers, the final fusion layer is obtained. The loss formula for the fusion layer is:

[0155] L fuse (X,Y,W)=L′ WCE (X,Y,W)+L Dice (X,Y,W)

[0156] The formula for the overall loss function of network model training is:

[0157] L = L side (X,Y,W,w)+L fuse (X,Y,W).

Claims

1. A method for operating a high-rise building exterior wall defect identification system based on unmanned aerial vehicles (UAVs), characterized in that, Carried by drone The camera completes the trajectory planning, and then... The camera identifies and classifies defects on the exterior walls of high-rise buildings. The operational steps are as follows: Step (1): The UAV uses its onboard lidar to communicate with... The camera scans and models the high-rise building environment, acquires point cloud data, and sets target points; Step (2): In the process of UAV path planning, an improved method is adopted. Path planning is performed, which uses path smoothing methods and constraint functions to constrain the UAV trajectory in order to obtain the shortest and safest UAV flight path; The improved The specific method for path planning is as follows: First, the initial population is defined as follows: It is the first The middle generation The horned lizard, introducing chaotic mappings The mapping, which utilizes its randomness and ergodicity to enhance search capabilities, also calculates the fitness of each horned lizard. The mathematical expression for the mapping is: In the formula, For iteration time, , It is an adjustable parameter. ; The proportion of the population to the maximum possible population size at time 1; join in The mapped individual horned lizards are updated using the following formula: The goal is to ensure that the initial population is evenly distributed in space and that the obtained initial population is closer to the optimal solution. When the horned lizard detects danger, it will choose different strategies depending on the situation. First, it will establish a stealth strategy and define... It is in the In the space of the generation A search agent, It is the first The optimal search agent for each generation is calculated using the following formula: ; In the formula, and It is a random integer generated between 1 and the maximum number of search agents, and satisfies... ; Represents the maximum number of iterations; The binary number is determined by checking if the random number from the color table is less than or equal to 0.5; if it is, it is 1, otherwise it is 0. Set to 2. and It is a simulated chromaticity and ; Furthermore, the improved horned lizard algorithm abandons the attack strategy and chooses to create a mobile escape strategy to improve the drone's handling of obstacles. The formula is as follows: In the formula, It is a random number generated between -1 and 1; These are random numbers generated by the standard Cauchy distribution, where Set to 0 and 1; Replace the worst search agent with a skin brightening and darkening strategy, using the following formula: In the formula, It is the first The search agent with the worst adaptability; and It is a random number generated between 0 and 0.4046661; and It is a random number generated between 0.5440510 and 1; To avoid premature convergence and escaping local optima, a random difference mutation strategy is introduced. This strategy uses the position of the current best individual and the position of a randomly selected individual to perform random difference processing to generate a new position. The specific formula is as follows: In the formula, Indicates within the range Random numbers between; Indicates in The newest individual in the generation; Indicates the first Random individuals in a generation; Smooth the planned path using... The algorithm handles redundant transformations and edges / corners in the path. The spline curve is: In the formula, These are the vertices of the characteristic polygon, formed by... Point-based splines are used to construct splines. for Step Secondary basis functions, substituted The result was: ; Set up constraint functions for the UAV, introducing performance constraints on the UAV's yaw and pitch angles, and filter out some successor nodes; let the current node of the UAV be... The next node selected by the drone is The previous parent node is Its deflection angle With pitch angle The formula is as follows: In the formula, For nodes 3D coordinates; For the next node 3D coordinates; The previous parent node 3D coordinates; ; The set of constraints for drones when selecting nodes is represented as follows: In the formula, For point The set of all successor nodes, and Indicates the maximum heading and pitch angles affected by the performance of the drone; Step (3): When the UAV is flying along the planned path, whenever the UAV reaches a target point, it uses the onboard... The camera photographs the defects on the exterior walls of high-rise buildings, using novel techniques. The algorithm identifies and classifies defects in the exterior walls of high-rise buildings; The novel approach is adopted The specific method by which the algorithm identifies and classifies defects in the exterior walls of high-rise buildings is as follows: Drone Improvement The output shows the optimal flight path, and each time the drone reaches a target point, it uses the onboard... The camera takes pictures of the exterior walls of high-rise buildings; a dataset of images of high-rise building exterior wall defects is created based on the collected images, including horizontal cracks and vertical cracks. Use union The algorithm identifies and classifies defects in the exterior walls of high-rise buildings, and combines them with... The algorithm's recognition process is as follows: The algorithm enhances the collected images of high-rise building exterior wall defects by improving... The model obtains accurate crack segmentation images and performs binarization. Then, morphological filtering is used to denoise and smooth the binarized crack images. Finally, a model is constructed. The mathematical model of the network is used to classify and identify cracks and fissures in high-rise buildings; Utilizing the improved The model obtains and binarizes the crack segmentation map, and then integrates multi-scale feature predictions with a neural network to improve performance. Network, Improved The network retains the original convolutional kernel pooling part and applies this fused prediction to... Characteristics of the network middle layer; It can obtain contextual information under different situations and enhance the network's ability to understand and judge target features; The output formula for this multi-scale feature prediction fusion is expressed as follows: In the formula, This is represented as the output after feature prediction fusion. These are represented as upsampled output feature maps at different scales.

2. The operation method of the high-rise building exterior wall defect identification system based on UAVs according to claim 1, characterized in that, In step (1), the lidar and The specific method for using cameras to collect environmental information from high-rise buildings is as follows: According to lidar and Camera definition acquisition settings A set of target points: And the initial position of the drone: ; The path planning sequence for the UAV is defined based on the lidar: In the formula, Indicates that the drone is in The location at the time of the step.

3. The operation method of the high-rise building exterior wall defect identification system based on UAVs according to claim 1, characterized in that, In step (3), Algorithms enhance images and perform multi-scale color enhancement. The algorithm performs multi-scale decomposition on the image, and performs [the following steps] on the image at each scale. The algorithm is enhanced to obtain reflectivity and luminance components, and then color correction and weighted synthesis are performed. The algorithm in During the enhancement process, different parameters are used for images of different scales, and the colors of the images are kept unchanged during the enhancement process, without causing oversaturation or color cast. The formula is expressed as follows: In the formula, This represents the gain value. The magnitude of the nonlinearity is related to the intensity of the nonlinearity.

4. The operation method of the high-rise building exterior wall defect identification system based on UAV according to claim 1, characterized in that, In step (3), the design of the loss function affects the performance of the network model. In the past, most image segmentation tasks used cross-entropy as the target loss function. The cross-entropy loss function is used for pixel-level classification. For crack images, the target crack accounts for a small proportion, so it is introduced that... Loss function; Considering the cross-entropy loss function and The advantages and disadvantages of loss functions are discussed. A weighted sum of the two loss functions yields a new loss function; the formula for the new loss function is: In the formula, Number of pixels Indicates the first A label of 1 pixel, Indicates the first Prediction results for each pixel; This is a hyperparameter to prevent the denominator from being 0; These are the weighting coefficients; Finally, the obtained crack segmentation map is binarized.

5. The operation method of the high-rise building exterior wall defect identification system based on UAV according to claim 4, characterized in that, Morphological filtering is used to denoise and smooth the image after the crack binarization. Morphological filtering constructs specific structural elements to perform front-to-back translation matching or local correction on the original signal, suppressing noise while preserving the morphological features of the signal. It eliminates isolated small points or small connected areas through corrosion expansion, while also eliminating burrs and uneven parts; The opening and closing operations in morphological processing are selected for filtering binarized images; the opening and closing operations are mainly based on the combination of erosion and dilation operators, and the formula is: In the formula, Indicates the object to be processed; For structural elements; It is an expansion symbol; The symbol is for corrosion. Images of high-rise building defects and cracks are divided into test and training sets for input. In the model, training The mathematical model of the network is used to classify and identify cracks and fissures in high-rise buildings.

Images