A bridge quality survey method based on quadrotor drone

By using a quadrotor drone to carry a high-resolution camera in bridge detection, combined with .NET track planning algorithm, ISSA algorithm and YOLO-v8 algorithm, the problem of long and low accuracy of bridge detection is solved, and high efficiency is achieved , accurate identification of bridge diseases.

CN117928539BActive Publication Date: 2024-11-05HUAIYIN INSTITUTE OF TECHNOLOGY
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202311686710.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-12-08
Publication Date
2024-11-05
Estimated Expiration
2043-12-08

AI Technical Summary

Technical Problem

The existing bridge detection technology has problems such as too long detection time and insufficient detection accuracy, making it difficult to effectively evaluate the surface diseases of the bridge body.

Method used

The bridge mass survey method based on quadrotor UAV is adopted, and the drone track planning is optimized by introducing .NET track planning algorithm and improved ISSA algorithm, and the bridge surface cracks are carried out in combination with YOLO-v8 algorithm and CBAM attention mechanism. and the identification of corrosion information to improve detection accuracy and efficiency.

Benefits of technology

It realizes efficient track planning and accurate disease identification of drones in bridge detection, significantly improves detection accuracy and efficiency, and can quickly and accurately identify cracks and corrosion phenomena on the bridge surface.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117928539B_ABST
    Figure CN117928539B_ABST
Patent Text Reader

Abstract

The present invention discloses a bridge quality survey method based on a quad-rotor drone, and the implementation process is as follows: the quad-rotor drone is equipped with a camera to collect bridge data, and modeling is completed according to the overall data of the bridge. The quad-rotor drone adopts the optimal track planning based on the .NET track planning algorithm for bridge quality survey, and at the same time adopts the ISSA algorithm to improve the track planning algorithm based on the .NET, so that the track of the quad-rotor drone is more accurate. The quad-rotor drone is equipped with a high-resolution camera (H20) to collect bridge data, complete the mathematical modeling of the bridge and the acquisition of bridge surface images; the collected bridge cracks and corrosion images are identified and marked using the YOLO‑v8 algorithm, and its loss function is improved to avoid being affected by the background texture of the bridge, resulting in errors in crack accuracy, and at the same time introduce the CBAM attention mechanism to enhance the algorithm's ability to extract features. The present invention proposes a method for the quad-rotor drone to autonomously complete bridge quality survey and identification, improve the accuracy of bridge quality survey, and save labor costs.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of bridge body detection, and in particular to a bridge body quality survey method based on a quad-rotor unmanned aerial vehicle. Background Art

[0002] Bridges are an important part of infrastructure, providing a guarantee for vehicles to pass across rivers, seas and buildings. The level of bridge construction is also a direct reflection of a country's comprehensive national strength. In recent years, bridge safety accidents have occurred frequently, posing a threat to people's lives and property safety and causing a great impact on society. Most of the bridges currently in service in China are reinforced concrete bridges. Detecting the apparent defects of concrete bridges is the main way to check the health of bridges.

[0003] Bridge diseases represent different degrees of severity according to their own conditions and the location of the components. According to the different characteristics of the diseases, they can be roughly divided into two categories: cracks and corrosion. Concrete cracks are multi-branched, long-extended small cracks on the surface, which are the main reason for the reduction of the bearing capacity, durability and waterproofness of concrete structures. Corrosion is manifested as the appearance of local crispness, pitting, small pits, crack expansion, etc. in the concrete structure to form a rough surface.

[0004] In terms of classic image processing technology, there are mainly methods based on grayscale threshold segmentation, edge detection and combination of specific tools. The detection methods such as formulating drone flight strategies in advance, collecting multiple overlapping images of a single span of the bridge, building a point cloud model to evaluate the specific defects of the bridge, and comparing some of the contents with the results of finite element analysis all have problems such as long detection time and insufficient detection accuracy.

[0005] The present invention takes the bridge surface disease phenomenon as the research object, proposes an improvement on the flight algorithm of the flying main body quad-rotor drone to enhance its flight effect. An improved image training algorithm is proposed for the bridge surface image data to improve the training speed and accuracy. Summary of the invention

[0006] Purpose of the invention: In view of the problems of long detection time and insufficient detection accuracy pointed out in the background technology, the present invention proposes a bridge quality survey method based on a quad-rotor drone, which improves the flight efficiency of the drone and the accuracy of bridge surface image recognition.

[0007] Technical solution: The present invention provides a bridge quality survey method based on a quad-rotor drone, which is implemented based on a quad-rotor drone equipped with a high-resolution camera, and includes the following steps:

[0008] Step 1: After the quadrotor drone takes off, the onboard H20 camera collects bridge data, generates an overall mathematical model of the bridge, and completes the track planning of the bridge according to the .NET track planning algorithm;

[0009] Step 2: Introduce the ISSA algorithm to improve the .NET trajectory planning algorithm to optimally solve the real-time position information of the UAV to avoid falling into the local optimal path. The ISSA algorithm introduces an adaptive perturbation strategy to accelerate the convergence speed of the trajectory range, improve the accuracy of the optimal solution, and complete the given task;

[0010] Step 3: During the quadrotor drone flight process in step 2, the onboard H20 camera collects bridge surface data;

[0011] Step 4: The YOLO-v8 algorithm is used to classify and identify cracks and corrosion information on the bridge surface, and improvements are made to the loss function. The confidence loss function, classification loss function, bounding box loss function, and prediction probability deviation function are used for weighted sum. At the same time, the CBAM attention mechanism is introduced to enhance the algorithm's ability to extract cracks and corrosion information on the bridge surface.

[0012] Step 5: Output feature image containing feature information.

[0013] Furthermore, the H20 camera in step 1 collects bridge data, and the implementation steps of generating a mathematical model of the bridge are as follows:

[0014] The panoramic information of the bridge is captured by the camera. The overlap rate of each picture taken by the camera at two points is ≥60%. Based on the panoramic data captured by the camera, a mathematical model is constructed to plan the UAV track. The pixel resolution of the H20 camera is ≤0.1mm.

[0015] According to the flight path of the UAV, the number of camera shots is increased during the process to ensure that there are overlapping parts in the captured pictures, so as to conduct a large-area problem area search; the UAV conducts a comprehensive inspection of the bridge, and uses oblique photogrammetry to establish a three-dimensional real-scene model of the UAV inspection part. The bridge body is divided into areas using a monomer method, and the divided areas are numbered. The GIS function is used to mark the area information involved in each section of the bridge body.

[0016] Furthermore, in step 1, the track planning of the bridge is completed according to the .NET track planning algorithm, and the specific implementation steps are:

[0017] Step 1: Construct the bridge space model;

[0018] Step 2: Determine the mathematical model for route planning based on UAV performance indicators and the performance of links and payloads;

[0019] Step 3: Obtain a flight path of the quadrotor drone by embedding the .NET trajectory planning algorithm;

[0020] Step 4: Interpolate and smooth the obtained track, adjust the track space model, and improve or re-plan the route;

[0021] Step 5: Output the final trajectory planning data and issue instructions to the drone to perform flight operations;

[0022] Step 6: The drone arrives at the target mission point to perform the mission.

[0023] Furthermore, in step 2, the ISSA algorithm is introduced to improve the .NET trajectory planning algorithm, and an adaptive perturbation strategy is introduced. The specific method is as follows:

[0024] In the SSA algorithm, the sparrow population N has explorers, predators and guards. The explorer is responsible for searching for the best individual. After the explorer finds the best individual, the predator preys on the best individual, and the guard is responsible for guarding the surroundings to avoid falling into the optimal solution. In the process of denoising the feature matrix SSA, the position of the explorer is updated as follows:

[0025]

[0026] Among them, a is the number of updates at time t; j is the current dimension of the sparrow population; is the jth dimension of the i-th sparrow when the data is updated to the a-th time at time t; iter max is the maximum number of iterations; α and Q are random numbers; T and ST are warning values ​​and safety thresholds, respectively; L is a d-dimensional unit vector;

[0027] The predator's position is updated as follows:

[0028]

[0029] in, is the worst solution of the ath iteration; is the optimal dimension of the explorer in the a+1th iteration; A is a 1×d-dimensional matrix, A + =A T (AA T ) -1 ;

[0030] The positions of the Sentinels have been updated as follows:

[0031]

[0032] in, is the optimal solution of the ath iteration; f i is the fitness value of the solution, f g is the fitness value of the optimal solution, f w is the fitness value of the worst solution; λ and J are random numbers; ε is a random constant;

[0033] An adaptive perturbation strategy is proposed to perturb individuals with better fitness values ​​in the tracking stage. The adaptive perturbation function formula is as follows:

[0034] Δp id =p id +η i

[0035] Among them, p id is the optimal position of individual i in the tth iteration; Δp id For p id The disturbance correction position of i ∈(-ζ d ,ζ d ) is the correction value of individual i in the tth iteration process, ζ d The calculation formula is:

[0036]

[0037] Where t is the current iteration number; is the average value of the optimal target position searched in t-1 iterations; c i is the proportional coefficient; Δr is the search area width; c is the adjustment constant.

[0038] Furthermore, in step 4, an improvement is proposed on the loss function, and the specific steps are as follows:

[0039] The YOLO-v8 algorithm loss function is a weighted sum of the following measurement functions:

[0040] The first type: confidence loss function, which is the intersection-over-union ratio of the target bounding box predicted by the network and the real bounding box for bridge cracks and corrosion phenomena. The calculation formula is as follows:

[0041]

[0042]

[0043] Among them, i ∈[0,1] is the IOU between the target real boundary and the predicted boundary; c i is the predicted value; c i The confidence prediction value through the Sigmoid function; N is the number of samples;

[0044] The second type: classification loss function. When cracks and corrosion are detected in a certain area of ​​the bridge body at the same time, the output labels are mutually exclusive. The calculation formula is as follows:

[0045]

[0046]

[0047] Among them, O ij ∈{0,1} indicates whether there is a j-th target in the predicted target bounding box i. If so, O ij =1, otherwise, O ij =0; C ij is the predicted value; C ij The target probability obtained by the Sigmoid function; N pos is the number of positive samples;

[0048] The third type: the bounding box loss function predicts the bounding box so that the predicted value of the bounding box is close to the true label value. The calculation formula is as follows:

[0049]

[0050] The coordinates (x, y) of the center point of the bounding box and the width and height of the bounding box are calculated as follows:

[0051]

[0052]

[0053]

[0054]

[0055] Among them, t x ,t y ,t w ,t h is the regression parameter predicted by the network; N pos is the total number of positive samples;

[0056] The fourth type: prediction probability deviation function, which reduces the deviation between the predicted value and the actual value of bridge cracks and corrosion phenomena. The calculation formula is as follows:

[0057]

[0058] Among them, p i (c) is the predicted value of the target; is the target true label value;

[0059] The improved YOLO-v8 algorithm loss function is calculated as follows:

[0060] L loss =α 1 Lconf +α 2 L cla +α 3 L loc +α 4 L obj

[0061] Among them, the calculation formula of Sigmoid function is as follows:

[0062]

[0063] Furthermore, the specific operation of introducing the CBAM attention mechanism in step 4 is:

[0064] In order to avoid interference from external environmental factors in the process of capturing bridge cracks and corrosion phenomena, the ACmix attention mechanism is proposed to fully mine the information of bridge cracks and corrosion phenomena and reduce the interference of other targets. The ACmix attention mechanism is implemented as follows:

[0065] Generate the entire feature map according to any pixel point in the convolution kernel, and get the feature map of the convolution kernel pixel of the convolution layer. The calculation formula is as follows:

[0066]

[0067] Among them, (p,q)∈{0,1,2…k-1} represents the convolution kernel position of the feature map; (i,j) is the pixel corresponding to the feature map; K (p,q) is the weight of the (p,q) position; f (i.j) is the feature tensor corresponding to pixel (i, j) of the input feature map; g (i,j) is the feature tensor corresponding to pixel (i, j) of the output feature map;

[0068] When outputting the eigenvalue, take the pixel value of the corresponding offset of each convolution kernel feature map and weight it to get the pixel eigenvalue of point (ij). The calculation formula is as follows:

[0069]

[0070]

[0071] Among them, the function of the Shift operator is to fuse multiple output feature maps. The calculation formula of the Shift operator is:

[0072]

[0073] Among them, Δx, Δy are the horizontal and vertical displacements corresponding to pixel (i, j).

[0074] Furthermore, the specific steps of using the YOLO-v8 algorithm in step 4 to classify and identify the cracks and corrosion information on the bridge surface are as follows:

[0075] Step 1: Divide the image captured by the H20 camera into s×s grids of equal size;

[0076] Step 2: Predict B bounding boxes for each grid;

[0077] Step 3: Predict the location coordinates of each bounding box;

[0078] Step 4: Predict the confidence value score for each bounding box;

[0079] Step 5: Predict the conditional category probability in each bounding box;

[0080] Step 6: Use the neural network structure of the YOLO-v8 algorithm to train the input and output images;

[0081] Step 7: Calculate the loss function for the target object in the grid;

[0082] Step 8: Output feature image containing feature information.

[0083] Beneficial effects:

[0084] 1. When the .NET trajectory planning algorithm of the quad-rotor UAV of the present invention performs optimal trajectory planning, an improved ISSA algorithm is introduced to optimally solve the real-time position information of the UAV, so that the UAV obtains more accurate target coordinate position information during the trajectory process, and an adaptive perturbation strategy is proposed to perform perturbation processing on individuals with better fitness values ​​in the tracking stage, and optimize the real-time position information of the UAV, so that the UAV obtains more accurate target coordinate position information during the trajectory process.

[0085] 2. The present invention uses the YOLO-v8 algorithm to automatically identify cracks and corrosion in bridge image data. To address the identification of special phenomena of bridge cracks and corrosion, the present invention proposes improvements to the loss function of the YOLO-v8 algorithm. The loss function is the weighted sum of the confidence loss function, the classification loss function, the bounding box loss function, and the prediction probability deviation function. The ACmix attention mechanism is proposed to fully mine the information of bridge cracks and corrosion phenomena, reduce the interference of other targets, and improve the detection accuracy of the improved YOLO-v8 algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] Figure 1 A system flow chart of a bridge quality survey method based on a quad-rotor drone provided in an embodiment of the present invention;

[0087] Figure 2Flowchart for training the YOLO-v8 algorithm on data collected by the H20 camera on a quadrotor drone;

[0088] Figure 3 Path planning process for the trajectory system of a quadrotor drone.

[0089] Figure 4 This is the extraction diagram of bridge cracks and corrosion phenomena using the improved YOLO-v8 algorithm proposed in the present invention. DETAILED DESCRIPTION

[0090] The present invention and the retrograde are further described below in conjunction with the drawings in the examples of the present invention. The following examples provide a detailed and complete description of the technical solution of the present invention, but do not limit the present invention in any form. On the basis of the present invention, a number of optimizations and improvements made by ordinary technicians in this field without making other creative ideas. The above all belong to the scope of protection of the present invention.

[0091] The present invention proposes a bridge quality survey method based on a quad-rotor drone. The implementation of this scheme is to use a quad-rotor drone equipped with a high-resolution camera to collect bridge data, and then classify and identify it. The quad-rotor drone is powered by a 12.6V aircraft model battery, four DC motors control the four wings of the drone to achieve flight in space, the IIC communication module completes the connection between the master device STM32 and the slave device MPU6050, the slave device MPU6050 obtains the attitude information of the drone during flight, and the master device STM32 reads the slave device information and sends instructions to the motor. The high-resolution camera model carried by the quad-rotor drone is an H20 camera.

[0092] The specific implementation scheme of a bridge quality survey method based on a quad-rotor drone disclosed in the present invention is carried out with reference to the following steps:

[0093] S1: Start the quad-rotor drone, the aircraft model battery starts to power each module of the drone, the motor drives the four wings to start working, and the drone is driven to any coordinate position in the space range by controlling the four wings to complete the multi-directional detection of the three-dimensional bridge by the onboard camera.

[0094] S2: To ensure that the quadrotor drone can autonomously complete the multi-directional survey task of the bridge, the quadrotor drone collects bridge modeling data by carrying an H20 camera to obtain panoramic picture information of the bridge. The overlap rate of each picture taken by the camera is ≥60%. Based on the panoramic data taken by the camera, a mathematical model is constructed to plan the drone's trajectory.

[0095] During the UAV tracking process, the number of camera shots is increased to ensure that there are overlapping parts in the captured pictures, so as to search for large problem areas; the UAV conducts a comprehensive inspection of the bridge and uses oblique photogrammetry to establish a three-dimensional real-scene model of the UAV inspection part.

[0096] S3: According to the mathematical model of the bridge, the present invention uses a .NET-based trajectory planning algorithm to complete the optimal trajectory planning of the quad-rotor UAV, ensuring that the UAV can arrive at the mission target coordinate point safely and on time to complete the established mission.

[0097] The implementation steps of the .NET-based trajectory planning algorithm used in the trajectory planning of the quadrotor drone are as follows:

[0098] Step 1: Construct the spatial model of the bridge.

[0099] Step 2: Determine the mathematical model for route planning based on UAV performance indicators and the performance of links and payloads.

[0100] Step 3: Obtain an optimal trajectory through the .NET-based trajectory planning algorithm.

[0101] Step 4: Output the final trajectory planning data and issue instructions to the drone to perform flight operations.

[0102] Step 5: The quadcopter flies along the tracked route.

[0103] S4: The present invention adopts an improved sparrow search algorithm (ISSA) algorithm to optimize the optimal trajectory planning for bridge quality survey based on the .NET trajectory planning algorithm, and optimizes the real-time position information of the UAV by improving the ISSA algorithm, so that the UAV can obtain more accurate target coordinate position information during the trajectory process.

[0104] In the SSA algorithm, the sparrow population N has explorers, predators and guards. The explorer is responsible for searching for the best individual. After the explorer finds the best individual, the predator preys on the best individual, and the guard is responsible for guarding the surroundings to avoid falling into the optimal solution. In the process of denoising the feature matrix SSA, the position of the explorer is updated as follows:

[0105]

[0106] Among them, a is the number of updates at time t; j is the current dimension of the sparrow population; is the jth dimension of the i-th sparrow when the data is updated to the a-th time at time t; iter max is the maximum number of iterations; α and Q are random numbers; T and ST are warning values ​​and safety thresholds, respectively; L is a d-dimensional unit vector;

[0107] The predator's position is updated as follows:

[0108]

[0109] in, is the worst solution for the ath iteration; is the optimal dimension of the explorer in the a+1th iteration; A is a 1×d-dimensional matrix, A + =A T (AA T ) -1 ;

[0110] The positions of the Sentinels have been updated as follows:

[0111]

[0112] in, is the optimal solution of the ath iteration; f i is the fitness value of the solution (f g is the fitness value of the optimal solution, f w is the fitness value of the worst solution); λ and J are random numbers; ε is a random constant and is small enough.

[0113] In the optimization process of the SSA algorithm, after a certain number of iterations, the search area of ​​the entire population will automatically converge to the optimal solution set. In order to accelerate the convergence speed of the search area and improve the accuracy of the optimal solution, the present invention proposes an adaptive perturbation strategy to perform perturbation processing on individuals with better fitness values ​​in the tracking stage. The adaptive perturbation function formula is as follows:

[0114] Δp id =p id +η i

[0115] Among them, p id is the optimal position of individual i in the tth iteration; Δp id For p id The disturbance correction position of i ∈(-ζ d ,ζ d ) is the correction value of individual i in the tth iteration process, ζ d The calculation formula is:

[0116]

[0117] Where t is the current iteration number; is the average value of the optimal target position searched in t-1 iterations; c i is the proportional coefficient; Δr is the search area width; c is the adjustment constant.

[0118] The implementation steps of the ISSA algorithm are:

[0119] Step 1: System initialization. Set the population size, maximum number of iterations, the ratio of search individuals to warning individuals, and the warning threshold of danger signals.

[0120] Step 2: Initialize population positions.

[0121] Step 3: Calculate the fitness value. Calculate the fitness value of individuals in the initial population and select individuals with high fitness values ​​to perform the search task.

[0122] Step 4: Update the location information of the searched individual.

[0123] Step 5: Update the location information of the tracked individual.

[0124] Step 6: Update the location information of the alerted individual.

[0125] Step 7: Introduce an adaptive perturbation strategy to perturb the individuals in the optimal position.

[0126] Step 8: Update the optimal individual position information and determine whether the maximum number of iterations has been reached. If so, output the optimal individual position information. Otherwise, return to step 3.

[0127] S5: The implementation steps of the optimal trajectory planning of the quadrotor drone for bridge quality survey based on the .NET trajectory planning algorithm improved by the ISSA algorithm are as follows:

[0128] Step 1: Construct the spatial model of the bridge.

[0129] Step 2: Determine the mathematical model for route planning based on UAV performance indicators and the performance of links and payloads.

[0130] Step 3: Obtain an optimal trajectory through the .NET-based trajectory planning algorithm.

[0131] Step 4: Use the SSA algorithm to optimally solve the real-time position information of the drone. At the same time, in order to avoid falling into the local optimum, an adaptive perturbation strategy is introduced to perturb the individuals in the optimal position.

[0132] Step 5: Perform interpolation and smoothing based on the obtained track, adjust the track space model, and improve or re-plan the route.

[0133] Step 6: Output the final trajectory planning data and issue instructions to the drone to perform flight operations.

[0134] Step 7: The quadcopter flies along the tracked route.

[0135] S6: During the track mission, the quadcopter drone uses an H20 camera to collect bridge surface inspection data. The bridge quality inspection is to set the camera pixel resolution to ≤ 0.1mm to ensure that the collected bridge cracks and corrosion images are clear. The bridge quality inspection method has two steps:

[0136] Step 1: Comprehensive inspection of the bridge structure; increase the number of camera shots during the drone tracking process to ensure that there are overlapping parts in the captured pictures and conduct a large-area problem area search; the drone conducts a comprehensive inspection of the bridge and uses oblique photogrammetry to establish a 3D real-life model of the drone inspection part. The bridge is divided into regions using a monomer method, and the divided regions are numbered. The GIS function is used to mark the areas involving cracks and corrosion information on each bridge.

[0137] Step 2: Fixed-point detection of bridge structure: For problem areas of the bridge, manual operation is used to allow the drone to reach the problem area for fixed-point data collection, and further detection is carried out based on the data collected at the fixed point.

[0138] S7: The collected image data uses an optimization model algorithm based on the YOLO framework to automatically identify cracks and corrosion. The YOLO algorithm is a target detection algorithm. The task of target detection is to find objects from images and give their categories and locations. When training the collected bridge crack and corrosion images, the training images are easily affected by the background texture of the bridge, resulting in unclear output features. The present invention proposes improvements to the loss function of the YOLO-v8 algorithm to cope with the identification of special bridge cracks and corrosion phenomena.

[0139] The improvement of the YOLO-v8 algorithm loss function is a weighted sum of the confidence loss function, the classification loss function, the bounding box loss function, and the prediction probability deviation function:

[0140] The first type: confidence loss function. The intersection-over-union ratio of the target bounding box predicted by the network and the real bounding box for bridge cracks and corrosion phenomena can better describe the improved YOLO-v8 target detection model. The calculation formula is as follows:

[0141]

[0142]

[0143] Among them, i ∈[0,1] is the IOU between the target real boundary and the predicted boundary; c i is the predicted value; c i Confidence prediction value through Sigmoid function; N is the number of samples.

[0144] Type 2: Classification loss function. When cracks and corrosion are detected in a certain area of ​​the bridge body at the same time, the output labels are mutually exclusive. The present invention proposes a target loss function to reduce the mutual exclusivity of cracks and corrosion. The calculation formula is as follows:

[0145]

[0146]

[0147] Among them, O ij ∈{0,1} indicates whether there is a target of type j in the predicted target bounding box i. If so, O ij =1, otherwise, O ij =0; C ij is the predicted value; C ij The target probability obtained by the Sigmoid function; N pos is the number of positive samples.

[0148] The third type: bounding box loss function. In target detection, the boundary of the framed target will be disturbed by factors such as light, resulting in the loss of the bounding box function. The bounding box loss function proposed in this invention predicts the bounding box so that the predicted value of the bounding box is close to the true label value. The calculation formula is as follows:

[0149]

[0150] The coordinates (x, y) of the center point of the bounding box and the width and height of the bounding box are calculated as follows:

[0151]

[0152]

[0153]

[0154]

[0155] Among them, t x ,t y ,t w ,t h is the regression parameter predicted by the network; N pos is the total number of positive samples.

[0156] The fourth type: prediction probability deviation function. The prediction probability deviation function is used to reduce the deviation between the predicted value and the actual value of bridge cracks and corrosion phenomena. The calculation formula is as follows:

[0157]

[0158] Among them, p i (c) is the predicted value of the target; is the target true label value.

[0159] The weighted sum of the above four loss functions is the improved YOLO-v8 algorithm loss function proposed in the present invention, and the calculation formula is as follows:

[0160] L loss =α 1 L conf +α 2 L cla +α 3 L loc +α 4 L obj

[0161] The Sigmoid function mentioned in the present invention is an activation function, which is used to provide nonlinear modeling capabilities for the neural network. The calculation formula of the Sigmoid function is as follows:

[0162]

[0163] The YOLO-v8 algorithm model is extremely sensitive to the information in the bridge image and is easily interfered by external environmental factors in the process of capturing cracks and corrosion phenomena on the bridge. The present invention proposes an ACmix attention mechanism to fully mine the information on cracks and corrosion phenomena on the bridge, reduce the interference of other targets, and improve the detection accuracy of the improved YOLO-v8 algorithm.

[0164] ACmix attention mechanism is implemented in two stages:

[0165] Stage 1: Generate the entire feature map based on any pixel point in the convolution kernel to obtain the feature map of the convolution kernel pixel of the convolution layer. The calculation formula is as follows:

[0166]

[0167] Among them, (p,q)∈{0,1,2…k-1} represents the convolution kernel position of the feature map; (i,j) is the pixel corresponding to the feature map; K (p,q) is the weight of the (p,q) position; f (i.j) is the feature tensor corresponding to pixel (i, j) of the input feature map; g (i,j) is the feature tensor corresponding to pixel (i, j) of the output feature map.

[0168] Stage 2: When outputting the eigenvalue, take the pixel value of the corresponding offset of each convolution kernel feature map and weight it to obtain the pixel eigenvalue of point (ij). The calculation formula is as follows:

[0169]

[0170]

[0171] Among them, the function of the Shift operator is to fuse multiple output feature maps. The calculation formula of the Shift operator is:

[0172]

[0173] Among them, Δx, Δy are the horizontal and vertical displacements corresponding to pixel (i, j).

[0174] For high-confidence cracks and corrosion areas identified by the improved YOLO-v8 algorithm, the present invention performs crack and corrosion segmentation on the identified images with characteristic values, and finally outputs an effect image.

[0175] The specific steps to improve the YOLO-v8 algorithm are:

[0176] Step 1: Divide the image captured by the H20 camera into s×s grids of equal size;

[0177] Step 2: Predict B bounding boxes for each grid;

[0178] Step 3: Predict the location coordinates of each bounding box;

[0179] Step 4: Predict the confidence value score for each bounding box;

[0180] Step 5: Predict the conditional category probability in each bounding box;

[0181] Step 6: Use the neural network structure of the YOLO-v8 algorithm to train the input and output images;

[0182] Step 7: Calculate the loss function for the target object in the grid;

[0183] Step 8: Output feature image containing feature information.

[0184] The embodiment of the present invention performs bridge survey. Figure 4 This is the effect diagram output after the improved YOLO-v8 algorithm extracts the crack and corrosion information of the bridge body. It can be seen from the figure that the present invention can clearly survey the damage of the bridge.

[0185] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with the technology to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent transformation or modification made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. A bridge quality survey method based on a quad-rotor drone, which is realized by using a quad-rotor drone equipped with a high-resolution camera, and is characterized in that: The following steps are involved: Step 1: After the quadrotor drone takes off, the onboard H20 camera collects bridge data, generates an overall mathematical model of the bridge, and completes the track planning of the bridge according to the .NET track planning algorithm; Step 1: Construct the bridge space model; Step 2: Determine the mathematical model for route planning based on UAV performance indicators and the performance of links and payloads; Step 3: Obtain a flight path of the quadrotor drone by embedding the .NET trajectory planning algorithm; Step 4: Interpolate and smooth the obtained track, adjust the track space model, and improve or re-plan the route; Step 5: Output the final trajectory planning data and issue instructions to the drone to perform flight operations; Step 6: The drone arrives at the target mission point to perform the mission; Step 2: Introduce the ISSA algorithm to improve the .NET trajectory planning algorithm to optimally solve the real-time position information of the UAV to avoid falling into the local optimal path. The ISSA algorithm introduces an adaptive perturbation strategy to accelerate the convergence speed of the trajectory range, improve the accuracy of the optimal solution, and complete the given task; In the SSA algorithm, the sparrow population N has explorers, predators and guards. The explorer is responsible for searching for the best individual. After the explorer finds the best individual, the predator preys on the best individual, and the guard is responsible for guarding the surroundings to avoid falling into the optimal solution. In the process of denoising the feature matrix SSA, the position of the explorer is updated as follows: Among them, a is the number of updates at time t; j is the current dimension of the sparrow population; is the jth dimension of the i-th sparrow when the data is updated to the a-th time at time t; iter max is the maximum number of iterations; α and Q are random numbers; T and ST are warning values ​​and safety thresholds, respectively; L is a d-dimensional unit vector; The predator's position is updated as follows: in, is the worst solution for the ath iteration; is the optimal dimension of the explorer in the a+1th iteration; A is a 1×d-dimensional matrix, A + =A T (AA T ) -1 ; The positions of the Sentinels have been updated as follows: in, is the optimal solution of the ath iteration; f i is the fitness value of the solution, f g is the fitness value of the optimal solution, f w is the fitness value of the worst solution; λ and J are random numbers; ε is a random constant; An adaptive perturbation strategy is proposed to perturb individuals with better fitness values ​​in the tracking stage. The adaptive perturbation function formula is as follows: Δp id =p id +n i Among them, p id is the optimal position of individual i in the tth iteration; Δp id For p id The disturbance correction position of i ∈(-ζ d ,ζ d ) is the correction value of individual i in the tth iteration process, ζ d The calculation formula is: Where t is the current iteration number; is the average value of the optimal target position searched in t-1 iterations; c i is the proportional coefficient; Δr is the width of the search area; c is the adjustment constant; Step 3: During the quadrotor drone flight process in step 2, the onboard H20 camera collects bridge surface data; Step 4: The YOLO-v8 algorithm is used to classify and identify cracks and corrosion information on the bridge surface, and improvements are made to the loss function. The confidence loss function, classification loss function, bounding box loss function, and prediction probability deviation function are used for weighted sum. At the same time, the CBAM attention mechanism is introduced to enhance the algorithm's ability to extract cracks and corrosion information on the bridge surface. The YOLO-v8 algorithm loss function is a weighted sum of the following measurement functions: The first type: confidence loss function, which is the intersection-over-union ratio of the target bounding box predicted by the network and the real bounding box for bridge cracks and corrosion phenomena. The calculation formula is as follows: Among them, i ∈[0,1] is the IOU between the target real boundary and the predicted boundary; c i is the predicted value; c i Confidence prediction value through Sigmoid function; N is the number of samples; The second type: classification loss function. When cracks and corrosion are detected in a certain area of ​​the bridge body at the same time, the output labels are mutually exclusive. The calculation formula is as follows: Among them, O ij ∈{0,1} indicates whether there is a target of type j in the predicted target bounding box i. If so, O ij =1, otherwise, O ij =0; C ij is the predicted value; C ij The target probability obtained by the Sigmoid function; N pos is the number of positive samples; The third type: the bounding box loss function predicts the bounding box so that the predicted value of the bounding box is close to the true label value. The calculation formula is as follows: The coordinates (x, y) of the center point of the bounding box and the width and height of the bounding box are calculated as follows: Among them, t x ,t y ,t w ,t h is the regression parameter predicted by the network; N pos is the total number of positive samples; The fourth type: prediction probability deviation function, which reduces the deviation between the predicted value and the actual value of bridge cracks and corrosion phenomena. The calculation formula is as follows: Among them, p i (c) is the predicted value of the target; is the target true label value; The improved YOLO-v8 algorithm loss function is calculated as follows: L loss =α1L conf +α2L cla +α3L loc +α4L obj Among them, the calculation formula of Sigmoid function is as follows: The specific operations of introducing the CBAM attention mechanism are: In order to avoid interference from external environmental factors in the process of capturing bridge cracks and corrosion phenomena, the ACmix attention mechanism is proposed to fully mine the information of bridge cracks and corrosion phenomena and reduce the interference of other targets. The ACmix attention mechanism is implemented as follows: Generate the entire feature map according to any pixel point in the convolution kernel, and get the feature map of the convolution kernel pixel of the convolution layer. The calculation formula is as follows: Among them, (p,q)∈{0,1,2…k-1} represents the convolution kernel position of the feature map; (i,j) is the pixel corresponding to the feature map; K (p,q) is the weight of the (p,q) position; f (i.j) is the feature tensor corresponding to pixel (i, j) of the input feature map; g (i,j) is the feature tensor corresponding to pixel (i, j) of the output feature map; When outputting the eigenvalue, take the pixel value of the corresponding offset of each convolution kernel feature map and weight it to get the pixel eigenvalue of point (ij). The calculation formula is as follows: Among them, the function of the Shift operator is to fuse multiple output feature maps. The calculation formula of the Shift operator is: Among them, Δx, Δy are the horizontal and vertical displacements corresponding to pixel (i, j); Step 5: Output feature image containing feature information.

2. The bridge quality survey method based on a quad-rotor drone according to claim 1 is characterized in that: The H20 camera in step 1 collects bridge data and generates a mathematical model of the bridge in the following steps: The panoramic information of the bridge is captured by the camera. The overlap rate of each picture taken by the camera at two points is ≥ 60%. Based on the panoramic data captured by the camera, a mathematical model is constructed to plan the UAV's track; H20 camera pixel resolution ≤ 0.1mm; According to the flight path of the UAV, the number of camera shots is increased during the process to ensure that there are overlapping parts in the captured pictures, so as to conduct a large-area problem area search; the UAV conducts a comprehensive inspection of the bridge, and uses oblique photogrammetry to establish a three-dimensional real-scene model of the UAV inspection part. The bridge body is divided into areas using a monomer method, and the divided areas are numbered. The GIS function is used to mark the area information involved in each section of the bridge body.

3. A bridge quality survey method based on a quad-rotor drone according to claim 1 or 2, characterized in that: In step 4, the YOLO-v8 algorithm is used to classify and identify the cracks and corrosion information on the bridge surface. The specific steps are: Step 1: Divide the image captured by the H20 camera into s×s grids of equal size; Step 2: Predict B bounding boxes for each grid; Step 3: Predict the location coordinates of each bounding box; Step 4: Predict the confidence value score for each bounding box; Step 5: Predict the conditional category probability in each bounding box; Step 6: Use the neural network structure of the YOLO-v8 algorithm to train the input and output images; Step 7: Calculate the loss function for the target object in the grid; Step 8: Output feature image containing feature information.

Citation Information

Patent Citations

  • Rapid crack detection method based on combination of improved YOLOv5 neural network and unmanned aerial vehicle video

    CN115223060A

  • Ship path optimization method, electronic equipment and storage medium

    CN115657693A

  • Bridge crack recognition and detection system and method based on unmanned aerial vehicle

    CN115937138A

  • Bridge concrete crack detection method based on YOLOv5 improved algorithm

    CN116485709A

  • Method and device for planning flight path of unmanned aerial vehicle under wind and rain conditions

    CN117029827A