Highway engineering quality monitoring and evaluating system based on intelligent unmanned aerial vehicle

Abstract

The invention relates to the technical field of highway engineering evaluation, in particular to a highway engineering quality monitoring and evaluating system based on an intelligent unmanned aerial vehicle. Defect classification is carried out through a convolutional neural network model, defect types are judged, crack detection, leakage detection and peeling detection are carried out on classified tunnel images, multi-direction gradients are introduced to calculate gradient intensities in different directions, fusion is carried out, and a fusion result is obtained. Jointly detecting a crack edge image in the tunnel image according to a fusion result and a local binary pattern, and calculating a crack width and a crack length based on the crack edge image; constructing a tunnel scoring model based on detection results of crack detection, leakage detection and spalling detection to calculate a tunnel quality score and a tunnel quality interval; and based on the current tunnel quality score and the collection time point, calculating a tunnel quality score and future risk time corresponding to the tunnel quality score in a subsequent level, and planning the latest collection time point of the unmanned aerial vehicle for collecting the tunnel image according to the future risk time.

Description

Technical Field

[0001] The present invention relates to the technical field of highway engineering evaluation, and particularly relates to a highway engineering quality monitoring and evaluation system based on intelligent unmanned aerial vehicles (UAVs). Background Art

[0002] With the continuous increase in traffic volume, continuous changes in environmental conditions, and the growth of service life, the structures and functions of tunnels are gradually facing different degrees of deterioration problems. These deterioration phenomena not only affect the service performance of tunnels but also pose potential threats to traffic safety.

[0003] Therefore, there is a current need to conduct quality monitoring and evaluation of tunnel highways (projects) to ensure timely response and control in the event of quality risks in tunnel highways. However, when conducting quality monitoring and evaluation of tunnel highways, the following problems still exist:

[0004] Currently, by collecting image data of tunnel highways and analyzing the damaged area and degree to judge the quality and health of tunnel highways, this method cannot accurately analyze the quality and health of tunnel highways in multiple dimensions;

[0005] Moreover, currently, by setting a fixed acquisition time for the UAV to achieve quality and health monitoring of tunnel highways at a specified time, due to the existence of the fixed acquisition time, if the tunnel highway is at risk of quality and health and requires repair and control during the time between the previous and the next acquisition times, this setting method of the acquisition time will affect the accuracy and timeliness of the UAV's monitoring and judgment of the quality and health of the tunnel highway, further affecting subsequent control and other treatments of the tunnel highway, and increasing the potential safety hazards of the tunnel highway operation. Summary of the Invention

[0006] In view of the above-mentioned drawbacks of the prior art, the present invention provides a highway engineering quality monitoring and evaluation system based on intelligent UAVs, which can effectively solve the problems in the prior art that it is impossible to evaluate tunnel highways in multiple dimensions and that the accuracy and timeliness of evaluating the quality and health of tunnel highways are affected by the current setting of fixed acquisition times.

[0007] To achieve the above objectives, the present invention is realized through the following technical solutions:

[0008] The present invention provides a highway engineering quality monitoring and evaluation system based on intelligent UAVs, which at least includes:

[0009] Including:

[0010] A tunnel image classification module, which collects tunnel images and performs defect classification through a convolutional neural network model to judge the defect types of tunnel images. Among them, the defect types include:

[0011] Cracks, leakage and spalling,

[0012] It also includes:

[0013] A tunnel evaluation module for respectively performing crack detection, leakage detection and spalling detection on the classified tunnel images. Among them, when performing crack detection:

[0014] Introduce multi-directional gradients to calculate the gradient intensities in different directions and fuse them to obtain a fusion result. Based on the fusion result and the local binary pattern, jointly detect the crack edge map in the tunnel image, and calculate the crack width and crack length based on the crack edge map;

[0015] The leakage detection is used to determine the centroid coordinates and the total leakage area of the leakage area;

[0016] The spalling detection is used to determine the spalling area and the spalling area shape factor;

[0017] Construct a tunnel scoring model based on the detection results of crack detection, leakage detection and spalling detection to calculate the tunnel quality score, and clarify the tunnel quality interval and the corresponding quality level;

[0018] If the tunnel quality interval is of good level, then based on the current tunnel quality score and the acquisition time point, calculate the tunnel quality score corresponding to the subsequent level and the future risk time, and plan the latest acquisition time point for the drone to acquire tunnel images according to the future risk time.

[0019] Furthermore, the steps for the convolutional neural network model to perform defect classification are:

[0020] Input the tunnel image;

[0021] Convolutional layer operation,

[0022]

[0023] Among them, F n,i,j,k represents the feature value at the position (i, j) in the feature map calculated by the k-th convolutional kernel in the convolutional layer for the n-th input tunnel image, f represents the activation function, and K u,v,c,k represents the weight of the k-th convolutional kernel at the position (u, v) and on the input tensor channel c, U and V respectively represent the height and width of the convolutional kernel, C represents the number of channels of the input tensor, K represents the number of convolutional kernels, and X n,i+u-1,j+v-1,c represents the pixel value at the position (i + u - 1, j + v - 1) of the n-th input tunnel image on c, and b k represents the bias of the k-th convolutional kernel;

[0024] Pooling layer operation,

[0025] P = Pool1(Fn,i,i,k )

[0026] Among them, P represents the pooled feature map, and Pool represents the pooling operation;

[0027] The high-dimensional features are mapped to the category space through the fully connected layer,

[0028] z k = WP + b

[0029] Among them, W represents the weight matrix, and z k represents the classification score of category k;

[0030] Perform classification output,

[0031]

[0032] Among them, represents the probability of category k, and z j represents the score of category j.

[0033] Furthermore, the method of introducing multi-directional gradients to calculate the gradient intensities in different directions and perform fusion is as follows:

[0034]

[0035] Among them, G x,θ (x, y), G y,θ (x, y) represent the gradients of the tunnel image in the x and y directions respectively, I(x, y) represents the input tunnel image, θ ∈ {0°, 45°, 90°, 135°}, respectively represent the filters in the x and y directions of the Sobel operator related to the direction θ;

[0036] For the gradients in each direction, calculate the corresponding gradient intensities, then there is:

[0037]

[0038] G θ (x, y) represents the gradient intensity of the tunnel image in the direction θ;

[0039] For each pixel position, calculate the gradient direction θ grad (x, y),

[0040] θ grad (x, y) = atan2(G y,θ (x, y)), G x,θ (x, y)

[0041] Among them, a tan2(y, x) represents the four-quadrant arctangent function that returns the point (x, y);

[0042] When fusing, we have:

[0043] G mut (x, y) = ω1·G 0° (x, y) + ω2·G 45° (x, y) + ω3·G 90° (x, y) + ω4·G 135° (x, y)

[0044] Among them, G mut (x, y) represents the fusion result of multi-directional gradients, and G 0° (x, y), G 45° (x, y), G 90° (x, y), and G 135° (x, y) respectively represent the gradient intensities of the tunnel image in the 0°, 45°, 90°, and 135° directions, and ω1, ω2, ω3, and ω4 are the corresponding weight coefficients.

[0045] Furthermore, the expression for the local binary pattern to extract the local texture features of the tunnel image is:

[0046]

[0047] Among them, LBP(x, y) represents the local binary pattern value calculated at the coordinate position (x, y), and I cenr (x, y) represents the gray value of the central pixel of the input tunnel image, and I neig (x p , y p ) represents the gray value of the neighborhood pixel in the input tunnel image, s(x) represents the sign function, 2 p represents the weighting factor corresponding to the neighborhood pixel, and p represents the total number of neighborhood pixels.

[0048] The method for the crack edge map to calculate the crack width and crack length is:

[0049] Based on the crack edge map F crack (x, y), calculate the width of each crack. For each crack, calculate the pixel distance of the crack edge:

[0050]

[0051] Among them, N crack represents the number of crack edge point pairs, and d edge (x i , 1, y i , 1, x i , 2, y i , 2,) represents the distance between the i-th pair of edge points;

[0052] Crack length:

[0053] The crack length is calculated by connecting the main axis points of the crack. Among them, the main axis points of the crack are obtained by performing a connected region analysis on the crack edge map F crack (x, y) to find all connected crack edge regions. Each connected region corresponds to a crack segment. Fit a minimum bounding rectangle or ellipse to each connected region, find the main axis direction of the crack, which is the corresponding main axis point, extract the key points on the main axis, determine the main axis points of the crack, and calculate the crack length L based on the main axis points crack , then there is:

[0054]

[0055] Among them, N line represents the number of main axis points of the crack, (x j , y j ) and (x j+1 , y j+1 ) are the coordinates of adjacent main axis points.

[0056] Furthermore, the method for determining the centroid coordinates and the total leakage area of the leakage area is as follows:

[0057] Calculate the total leakage area A of the leakage area through the leakage distribution map F leak (x, y), then there is: leak , then there is:

[0058]

[0059] Calculate the centroid coordinates of the leakage area, then there is:

[0060]

[0061] Among them, (x c , y c ) represents the centroid coordinates of the leakage area, represents the pixel weighted value of the leakage area.

[0062] Furthermore, a tunnel scoring model is constructed to evaluate the quality of the current tunnel project, then there is:

[0063] S ck =(W crack +L crack )β1+(A leak +distance(x c , y c ))β2+(A spall +S shape )β3

[0064] Among them, Sck Indicates the tunnel quality score, and β1······β3 respectively represent the corresponding weight coefficients. distance(x c ,y c ) represents the distance from the centroid in the leakage area to the tunnel structure. A spall represents the spalling area, and S shape represents the shape factor of the spalling area.

[0065] Furthermore, the tunnel quality intervals and the corresponding quality grades include excellent, good, average, and poor, where:

[0066] If the tunnel quality interval corresponds to the good grade, then obtain the current S ck , and obtain the multiple historical collection time points corresponding to S ck that are closest to the current collection time point at the corresponding collection time points. ck ;

[0067] Calculate the difference between S ck at adjacent collection time points and sum them, and calculate the average value to obtain the average difference. At the same time, calculate the time between adjacent collection time points and sum them, and calculate the average value to obtain the average time;

[0068] Based on obtain the difference change S′ per hour cg ;

[0069] Based on the difference between the current S ck and the S ck corresponding to the average grade tunnel quality interval, obtain the class difference S′ cx . Based on the ratio of obtain the estimated consumption time S′ xa ;

[0070] Calculate the collection time point corresponding to the current S xa and the estimated consumption time S′ ck to obtain the future risk time S′ ck corresponding to the S fx in the average grade tunnel quality interval of the tunnel quality score. Based on the future risk time S′ fx supervise the tunnel highway.

[0071] Furthermore, it also includes

[0072] Based on the future risk time S′ fx set the collection time point and determine the average collection time S c for the drone to collect tunnel images;

[0073] Determine the disposal loss time S′ for the tunnel highway to implement safety controlcs , there is:

[0074] Based on the future risk time S′ fx , the average acquisition time S c and the disposal loss time S′ cs Determine the latest acquisition time point S′ for tunnel image monitoring zw , based on the current time and the latest acquisition time point S′ zw Set the acquisition time point for acquiring tunnel images to implement tunnel highway control.

[0075] Furthermore, based on the latest acquisition time point S′ zw Form a planning restriction on the UAV acquisition path.

[0076] The technical solution provided by the present invention has the following beneficial effects compared with the known prior art:

[0077] By performing crack detection, leakage detection and spalling detection on the classified tunnel images, introducing multi-directional gradients to calculate the gradient intensities in different directions and fusing them, and jointly detecting the crack edge map in the tunnel image in combination with the local binary pattern, the accurate extraction of the crack edge map is realized. Thus, based on the crack edge map, the crack width and crack length are calculated for subsequent accurate judgment of the tunnel quality;

[0078] Based on the crack width, crack length, total leakage area, centroid coordinates of the leakage area, spalling area and spalling area shape factor, a tunnel scoring model is constructed to calculate the tunnel quality score, forming a multi-dimensional tunnel quality assessment to improve accuracy;

[0079] Based on the tunnel quality score and the acquisition time point, calculate the tunnel quality score and the future risk time corresponding to the general grade of the tunnel quality score. According to the future risk time, plan the latest acquisition time point for the UAV to acquire tunnel images, and define the latest time for the UAV to acquire tunnel images and analyze the tunnel quality in a timely manner, thereby avoiding the situation that the acquisition time point set currently will cause a delay in the tunnel quality risk assessment and reducing the tunnel risk potential. Description of the Drawings

[0080] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0081] Figure 1 It is the overall module block diagram of the present invention. Detailed Embodiments

[0082] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0083] The present invention will be further described below with reference to embodiments.

[0084] Embodiment 1 (refer to Figure 1 ): A highway engineering quality monitoring and evaluation system based on intelligent unmanned aerial vehicles (UAVs) includes at least:

[0085] A tunnel image classification module that collects tunnel images of tunnel roads through a UAV and preprocesses the tunnel images, such as denoising or filtering the images to remove noise caused by UAV flight vibration or light changes, and then classifies the defects of the collected tunnel images through a convolutional neural network model to clarify the defect types of the tunnel images. The steps for classifying the defects of the tunnel images by the convolutional neural network model are as follows:

[0086] Input the tunnel image;

[0087] Use a convolutional layer to extract the basic features of the tunnel image,

[0088]

[0089] where F n,i,j,k represents the feature value at the position (i, j) in the feature map calculated by the k-th convolutional kernel for the n-th input tunnel image in the convolutional layer, f represents the activation function used to introduce non-linearity, such as ReLU, to perform non-linear transformation on the convolutional result of the linear combination and enhance the expression ability of the model, K u,v,c,k represents the weight of the k-th convolutional kernel at the position (u, v) and on the input tensor channel c, U and V respectively represent the height and width of the convolutional kernel, C represents the number of channels of the input tensor, K represents the number of convolutional kernels, X n,i+u-1,j+v-1,c represents the pixel value at the position (i + u - 1, j + v - 1) on the c-th channel of the n-th input tunnel image, which is the input element corresponding to the position of the convolutional kernel during the convolution operation, and b k represents the bias of the k-th convolutional kernel;

[0090] Perform dimensionality reduction through a pooling layer to reduce the amount of calculation,

[0091] P = Pool(F n,i,i,k )

[0092] Among them, P represents the pooled feature map, and Pool represents the pooling operation, including max pooling or average pooling;

[0093] The high-dimensional features are mapped to the category space through the fully connected layer,

[0094] z k = WP + b

[0095] Among them, W represents the weight matrix, and z k represents the classification score of category k;

[0096] Perform classification output,

[0097]

[0098] Among them, represents the probability of category k, and z j represents the score of category j. Among them, for the defect categories of tunnel images, they include crack, leakage, and spalling defects. Therefore, the convolutional neural network model is used to identify and classify the defects, which is convenient for further detection of the defects.

[0099] Therefore, based on the defect classification of tunnel images, the tunnel images corresponding to crack, leakage, and spalling defects are obtained respectively. In order to facilitate the quality assessment of the tunnel highway, a tunnel assessment module is also included, which includes the following steps:

[0100] Perform crack detection:

[0101] Extract the crack edges of the tunnel image through edge detection operation. The crack edge map F crack (x, y), where the steps of the edge detection operation are:

[0102] Perform Gaussian smoothing on the input tunnel image to reduce the influence of noise on edge detection;

[0103] Relying on the Gaussian-smoothed tunnel image, combined with multi-directional gradients and local binary pattern (LBP), accurately detect the crack edges in the tunnel image, then there is:

[0104] In order to capture more complex crack morphologies, introduce gradient calculations in multiple directions,

[0105]

[0106] Among them, G x,θ (x, y), G y,θ (x, y) respectively represent the gradients of the tunnel image in the x and y directions, I(x, y) represents the input tunnel image, and θ ∈ {0°, 45°, 90°, 135°}, respectively represent the filters of the Sobel operator related to the direction θ in the x and y directions;

[0107] For the gradient in each direction, calculate the corresponding gradient intensity to clarify the amplitude of pixel change. Then,

[0108]

[0109] G θ G(x,y) represents the gradient intensity of the tunnel image in the direction θ, quantifying the edge saliency of the image in this direction;

[0110] For each pixel position, calculate the gradient direction θ grad (x,y),

[0111] θ grad θ(x,y) = atan2(G y,θ (x,y)), G x,θ (x,y)

[0112] where atan2(y,x) represents the four-quadrant arctangent function that returns the point (x,y), used to calculate the gradient direction;

[0113] After calculating the gradient intensities in different directions and performing fusion, then,

[0114] G mut G(x,y) = ω1·G 0° (x,y) + ω2·G 45° (x,y) + ω3·G 90° (x,y) + ω4·G 135° (x,y)

[0115] where G mut (x,y) represents the fusion result of multi-directional gradients, G 0° (x,y), G 45° (x,y), G 90° (x,y), and G 135° (x,y) respectively represent the gradient intensities of the tunnel image in the 0°, 45°, 90°, and 135° directions. ω1, ω2, ω3, and ω4 are the corresponding weight coefficients, which can be preset values. Furthermore, by calculating the gradient intensities in different directions, more edge information can be extracted, especially for tunnel images with complex crack morphologies.

[0116] Thus, based on the fusion result of the multi-directional gradients obtained above, introduce the Local Binary Pattern (LBP) to extract the local texture features of the tunnel image. LBP can effectively capture the local texture structure of the image. Then,

[0117]

[0118] Among them, LBP(x, y) represents the local binary pattern value calculated at the coordinate position (x, y), which is used to describe the texture features of the local area around this point, and I cenr (x, y) represents the gray value of the central pixel of the input tunnel image, which is used to compare with the surrounding pixels, and I neig (x p , y p ) represents the gray value of the neighborhood pixels in the input tunnel image, which is used to compare with the central pixel. s(x) represents the sign function, defined as s(x) = 1, which converts the gray difference of the pixel into binary 0 or 1 to represent the texture features at this position, and 2 p represents the weighted factor corresponding to the neighborhood pixel, which describes the weight of this pixel. p represents the total number of neighborhood pixels, that is, the number of pixels around the central pixel. By extracting the local texture features as above, the recognition of cracks and their surrounding textures is further enhanced;

[0119] Therefore, by performing feature fusion on the obtained G mut (x, y) and LBP(x, y), a more refined crack edge map F crack (x, y) is obtained. The method of weighting will not be elaborated here, and it can be achieved by assigning corresponding weight coefficients respectively;

[0120] After gradient calculation, non-maximum suppression is used to remove the pixel points that are not local maxima, and only the most significant pixel points in the edge direction are retained to refine the edge position and ensure that only the most significant edges are retained;

[0121] Double-thresholding processing is performed to classify the gradient intensity according to the set high threshold and low threshold, further removing the pixel noise below the low threshold, and retaining the true crack edge map F crack (x, y), which reflects the position and shape of the cracks in the tunnel image.

[0122] Crack width:

[0123] Based on the obtained crack edge map F crack (x, y), the width of each crack is calculated. For each crack, the pixel distance of the crack edge is calculated:

[0124]

[0125] Among them, N crack represents the number of crack edge point pairs, and d edge (x i , 1, y i , 1, x i , 2, y i , 2,) represents the distance between the i-th pair of edge points;

[0126] Crack length:

[0127] The crack length is calculated by connecting the main axis points of the crack. Among them, the main axis points of the crack are obtained by performing a connected region analysis on the crack edge image F crack (x, y), finding all connected crack edge regions. Each connected region corresponds to a possible crack segment. For each connected region, a minimum bounding rectangle or ellipse is fitted. This fitting process uses the principal component analysis (PCA) method to find the main axis direction of the crack (i.e., the longest axis of the crack) and the main axis points in this direction. From the fitted minimum bounding rectangle or ellipse, the key points on the main axis are extracted (for example, the endpoints or midpoints in the long axis direction). These points are the main axis points of the crack. Based on the main axis points, the length L of the crack crack is obtained by calculating the distances between these main axis points. Then,

[0128]

[0129] where N line represents the number of main axis points of the crack, (x j , y j ) and (x j+1 , y j+1 ) are the coordinates of adjacent main axis points.

[0130] Leakage detection:

[0131] Color component analysis,

[0132] The leakage area is detected by extracting the hue H(x, y) and saturation S(x, y) of each pixel. Set the hue threshold H range and the saturation threshold S range , and generate the leakage distribution map F leak (x, y);

[0133] According to the hue threshold H range and the saturation threshold S range , the pixel points in the tunnel image covering the leakage characteristics that meet the hue threshold H range and the saturation threshold S range are marked as 1, and those that do not meet are marked as 0;

[0134] Therefore, the total leakage area A of the leakage area is calculated through the leakage distribution map F leak (x, y), that is, the number of pixels in the leakage area. Then, leak ,

[0135]

[0136] where the above summation statistics Fleak The number of pixels with a value of 1 in (x, y);

[0137] Calculate the centroid coordinates of the leakage area and further analyze the spatial distribution of the leakage, then there is:

[0138]

[0139] Among them, (x c , y c ) represents the centroid coordinates of the leakage area, represents the pixel weighted value of the leakage area;

[0140] Perform spalling detection:

[0141] Shape segmentation: Segment the spalling area from the input image through a deep learning method (MaskRCNN) to generate a spalling area map F spall (x, y), where the position with a value of 1 represents the spalling area in the image, and the position with a value of 0 represents the non-spalling area;

[0142] Spalling area A spall Calculation: Calculate the spalling area by counting all the pixels belonging to the spalling area in the spalling area image, then there is:

[0143]

[0144] F spall (x, y) represents the pixel value of the spalling area image, where a value of 1 indicates that the pixel belongs to the spalling area, and 0 indicates that it does not belong to the spalling area;

[0145] Spalling area shape factor S shape Calculation: The spalling area shape factor is used to describe the shape characteristics of the spalling area. Usually, the ratio of the perimeter to the area of the spalling area is calculated. The formula is as follows:

[0146]

[0147] Among them, S shape represents the spalling area shape factor, P spall 2 represents the perimeter of the spalling area, which is calculated through an edge detection or contour extraction algorithm (such as a contour tracing algorithm), and will not be elaborated here;

[0148] Based on the above, by integrating the detected tunnel defects, a defect feature set {W crack , L crack , A leak , (x c , y c ), A spall , Sshape};

[0149] Therefore, by constructing a tunnel scoring model to evaluate the current tunnel engineering quality, we have:

[0150] S ck =(W crack +L crack )β1+(A leak +distance(x c ,y c ))β2+(A spall +S shape )β3

[0151] Among them, S ck represents the tunnel quality score, β1······β3 respectively represent the corresponding weight coefficients, which can be preset values, distance(x c ,y c ) represents the distance from the centroid of the leakage area to the tunnel structure (usually the center or edge of the tunnel);

[0152] Therefore, according to S ck to judge the corresponding tunnel quality interval (the larger S ck , the larger the corresponding tunnel quality interval, indicating that the quality of the current tunnel project is worse. Usually, the tunnel quality interval can be divided into four grades, such as excellent, good, general, and poor. When it is excellent, it means that the quality of the current tunnel project is good, that is, S ck is in a smaller tunnel quality interval and does not require treatment such as repair. When it is good, regular monitoring of the tunnel project is required. Both general and poor indicate that the quality of the current tunnel project needs to be repaired immediately), based on the tunnel quality interval, the following steps are executed:

[0153] If the tunnel quality interval corresponds to excellent, there is no need to treat the tunnel project. According to the historically preset acquisition time point, the drone can collect and analyze the tunnel images according to the acquisition time point;

[0154] If the tunnel quality interval corresponds to good, then obtain the current S ck , and obtain the S ck corresponding to the nearest multiple (at least two) historical acquisition time points under the acquisition time point corresponding to the current S ck , calculate the difference between the S ck at adjacent acquisition time points and sum them, and calculate the average value to obtain the average difference. At the same time, calculate the time between adjacent acquisition time points and sum them, and calculate the average value to obtain the average time. Corresponding the average time with the average difference to reflect the average difference between the upper and lower acquisition time points, based on (The unit of the average time is hours) Obtain the change in difference S' per hour cg , thereby obtaining the S corresponding to the tunnel quality interval of the general grade ck , thereby, based on the current S ck and the S corresponding to the tunnel quality interval of the general grade ck Take the difference between them to obtain the class difference S' cx , based on Obtain the predicted consumption time S' based on the ratio between them xa , add the predicted consumption time S' xa to the acquisition time point corresponding to the current S ck to obtain the S corresponding to the tunnel quality score of the tunnel project being in the tunnel quality interval of the general grade ck for the future risk time S' fx , which can indicate that at the future risk time S' fx this tunnel project will be in the tunnel quality interval of the general grade, so as to indicate that the management personnel of the tunnel highway need to execute supervision on the tunnel highway before the future risk time S' fx . For the convenience of executing supervision, it also includes:

[0155] Set an acquisition time point for the tunnel highway before the future risk time S' fx , and determine the average acquisition time based on the time when the drone has collected the current tunnel highway each time in history

[0156] At the same time, determine the disposal loss time for the management personnel to execute safety control on this tunnel highway. The disposal loss time S' can be determined according to the ratio of the distance between the location of the management personnel and the location of the tunnel highway and the speed cs . For the disposal loss time S' cs it means that when the tunnel highway is in the tunnel quality interval of the general grade, it needs to be repaired. Then, in order to improve the safety of the tunnel highway, it is necessary to pre-execute safety control (set risks, no-entry signs, etc.) on this tunnel highway before the repair. Therefore, in order to ensure the determination of the tunnel highway being in the tunnel quality interval of the general grade and be able to execute safety control in advance, there is:

[0157] Subtract the average acquisition time fx from the future risk time S' and the disposal loss time S' cs to re-determine the latest time for tunnel image monitoring of the tunnel highway and denote it as the latest acquisition time point S' zw , in this way, based on the current time and the latest acquisition time point S' zw ​Set the acquisition time point for the drone to acquire tunnel images within the time interval, ensuring that after the tunnel images are acquired by the drone and analyzed, if the tunnel highway is in the tunnel quality interval under the general grade, the control personnel can be pre-instructed to control the tunnel highway so as to perform corresponding tunnel highway repairs subsequently.

[0158] It should be noted that based on the determined latest acquisition time point S′ zw , when subsequently assigning the drone to acquire tunnel images, a time condition restriction can be generated during the planning of the drone's acquisition path. In this way, not only can the risk monitoring of the tunnel highway be achieved, but also the flight path planning (optimal) of the drone is facilitated, which is conducive to the drone performing its operation tasks.

[0159] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the various embodiments of the present invention.

Claims

1. Highway engineering quality monitoring and evaluation system based on intelligent drones, including: The tunnel image classification module collects tunnel images and classifies defects through a convolutional neural network model to determine the defect type of the tunnel image. The defect types include: Cracks, leaks and peeling, It is characterized by further comprising: The tunnel assessment module is used to perform crack detection, leakage detection and spalling detection on the classified tunnel images. When performing crack detection: Multi-directional gradients are introduced to calculate the gradient strengths in different directions and fuse them to obtain the fusion results. The crack edge map in the tunnel image is detected based on the fusion results and the local binary pattern, and the crack width and length are calculated based on the crack edge map. The leakage detection is used to determine the centroid coordinates of the leakage area and the total leakage area; The peeling detection is used to determine the peeling area and the peeling area shape factor; Based on the test results of crack detection, leakage detection and spalling detection, a tunnel scoring model is constructed to calculate the tunnel quality score, and the tunnel quality range and corresponding quality level are clearly defined; If the tunnel quality interval is good, the tunnel quality score and future risk time corresponding to the subsequent level of tunnel quality score are calculated based on the current tunnel quality score and collection time point, and the latest collection time point for the drone to collect tunnel images is planned according to the future risk time.

2. The highway engineering quality monitoring and evaluation system based on intelligent drone according to claim 1 is characterized in that: The steps of defect classification by the convolutional neural network model are as follows: Input tunnel image; Convolutional layer operation, Among them, F n,i,j,k represents the feature value of the position (i, j) in the feature map of the nth input tunnel image in the convolution layer calculated by the kth convolution kernel, f represents the activation function, K u,v,c,k represents the weight of the kth convolution kernel at position (u, v) and input tensor channel c. U and V represent the height and width of the convolution kernel, respectively. C represents the number of channels of the input tensor. K represents the number of convolution kernels. X n,i+u-1,j+v-1,c represents the pixel value of the nth input tunnel image at position (i+u-1,j+v-1) on c, b k Represents the bias of the kth convolution kernel; Pooling layer operation, P=P OO 1(F n,i,i,k ) Among them, P represents the feature map after pooling, P OO 1 indicates pooling operation; The high-dimensional features are mapped to the category space through the fully connected layer. With k =WP+b Where W represents the weight matrix, z k represents the classification score of category k; Perform classification output, in, represents the probability of category k, z j represents the score of category j.

3. The highway engineering quality monitoring and evaluation system based on intelligent drone according to claim 1 is characterized in that: The method of introducing multi-directional gradients to calculate the gradient strengths in different directions and perform fusion is: Among them, G x,θ (x,y),G y,θ (x, y) represents the gradient of the tunnel image in the x and y directions respectively, I(x, y) represents the input tunnel image, θ∈{0°, 45°, 90°135°}, S θx (i,j), Sθ y (i, j) represents the filter of the Sobel operator in the x and y directions associated with the direction θ, respectively; For the gradient in each direction, calculate the corresponding gradient strength, then: G θ (x, y) represents the gradient intensity of the tunnel image in the direction θ; For each pixel position, calculate the gradient direction θ grad (x,y), θ grad (x,y)=atan2(G y,θ (x,y)),G x,θ (x,y) Among them, atan2(y,x) means the four-quadrant inverse tangent function that returns the point (x,y); To merge, we have: G mut (x,y)=ω1·G 0° (x,y)+ω2·G 45° (x,y)+ω3·G 90° (x,y)+ω4·G 135° (x,y) Among them, G mut (x, y) represents the fusion result of multi-directional gradients, G 0° (x,y),G 45° (x,y),G 90° (x,y) and G 135° (x, y) represents the gradient intensity of the tunnel image in the directions of 0°, 45°, 90°, and 135°, respectively, and ω1, ω2, ω3, and ω4 are the corresponding weight coefficients.

4. The highway engineering quality monitoring and evaluation system based on intelligent drone according to claim 1 is characterized in that: The expression of extracting the local texture features of the tunnel image by the local binary pattern is: Where LBP(x,y) represents the local binary pattern value calculated at the coordinate position (x,y), I cenr (x, y) represents the gray value of the central pixel of the input tunnel image, I neig (x p ,y p ) represents the gray value of the neighborhood pixel in the input tunnel image, s(x) represents the sign function, 2 p represents the weighting factor corresponding to the neighborhood pixels, and p represents the total number of neighborhood pixels.

5. The highway engineering quality monitoring and evaluation system based on intelligent unmanned aerial vehicles according to claim 4 is characterized in that: The method for calculating the crack width and crack length of the crack edge map is: Based on the crack edge map F crack (x,y) Calculate the width of each crack, and for each crack, calculate the pixel distance to the edge of the crack: Among them, N crack represents the number of crack edge point pairs, d edge (x i ,1,y i ,1,x i ,2,y i ,2,) represents the distance between the i-th pair of edge points; Crack length: The crack length calculation is completed by connecting the main axis points of the crack, where the main axis points of the crack are calculated by the crack edge map F crack (x, y) is used to analyze the connected regions, find all connected crack edge regions, each connected region corresponds to a crack segment, fit a minimum circumscribed rectangle or ellipse to each connected region, find the main axis direction of the crack, that is, the corresponding main axis point, extract the key points where the main axis is located, determine the main axis point of the crack, and calculate the length L of the crack based on the main axis point. crack , then: Among them, N line represents the number of crack main axis points, (x j ,y j ) and (x j+1 ,y j+1 ) are the coordinates of the adjacent principal axis points.

6. The highway engineering quality monitoring and evaluation system based on intelligent drone according to claim 5 is characterized in that: The method for determining the centroid coordinates of the leakage area and the total leakage area is: Through the leakage distribution map F leak (x,y) Calculate the total leakage area A of the leakage area leak , then: Calculate the centroid coordinates of the leakage area, then: Among them, (x c ,y c ) represents the centroid coordinates of the leakage area, The pixel weight value representing the leakage area.

7. The highway engineering quality monitoring and evaluation system based on intelligent drone according to claim 6 is characterized in that: Construct a tunnel scoring model to evaluate the current tunnel engineering quality, then: S ck =(W crack +L crack )β1+(A leak +distance(x c ,y c ))β2+(A spall +S shape )β3 Among them, S ck represents the tunnel quality score, β1······β3 represent the corresponding weight coefficients, distance(x c ,y c ) represents the distance from the center of gravity in the leakage area to the tunnel structure, A spall represents the area of ​​the peeling region, S shape represents the spalling area shape factor.

8. The highway engineering quality monitoring and evaluation system based on intelligent drone according to claim 7 is characterized in that: The tunnel quality intervals and corresponding quality levels include excellent, good, average, and poor, where: If the tunnel quality interval corresponds to the good level, then obtain the current S ck , and obtain the distance from the current S ck The corresponding S at the corresponding collection time point at the latest multiple historical collection time points ck ; Calculate S at adjacent acquisition time points ck The differences between the two are summed up, and the average value is calculated to obtain the average difference. At the same time, the time between adjacent acquisition time points is calculated and summed up, and the average value is calculated to obtain the average time. based on Get the difference change S' per hour cg ; Based on the current S ck S corresponding to the general grade tunnel quality interval ck The difference between them is the class difference S′ cx ,based on The estimated consumption time S' is obtained by comparing xa ; The estimated time consumed is S' xa With the current S ck The corresponding acquisition time point is calculated to obtain the S corresponding to the tunnel quality score in the general grade tunnel quality interval. ck The future risk time S′ fx , based on the future risk time S′ fx Carry out supervision on tunnel roads.

9. The highway engineering quality monitoring and evaluation system based on intelligent drone according to claim 8 is characterized in that: Also includes, Based on the future risk time S′ fx Set the acquisition time point and determine the average acquisition time S of the UAV to collect tunnel images c ; Determine the disposal loss time S′ for executing safety control on the tunnel highway cs , then: Based on the future risk time S′ fx , average collection time and disposal loss time S′ cs Determine the latest acquisition time point S′ for tunnel image monitoring zw , based on the current time and the latest acquisition time point S′ zw Set the time point for collecting tunnel images to perform tunnel road control.

10. The highway engineering quality monitoring and evaluation system based on intelligent drone according to claim 9 is characterized in that: Based on the latest acquisition time point S′ zw Form planning restrictions on drone collection paths.

Images