A method and system for real-time inspection of road surface diseases based on a UAV

Abstract

The application relates to the technical field of road inspection, and discloses a road surface disease real-time inspection method and system based on an unmanned aerial vehicle, wherein the method comprises the following steps: collecting images of a road surface by using an unmanned aerial vehicle and performing pretreatment; using an improved YOLOv5 algorithm to detect disease targets in the pretreated images; using an improved convolutional neural network to extract features of the detected disease targets; using an improved Vision Transformer algorithm to classify and identify the extracted disease features and determine the type of the disease; after the type of the disease is determined, the severity of the disease is evaluated by using a three-dimensional reconstruction algorithm, and a disease report is generated. The application collects images by using an unmanned aerial vehicle, combines improved algorithms, improves road disease inspection efficiency and precision, realizes real-time feedback of disease information, generates a detailed report, provides a comprehensive basis for road maintenance, reduces costs and time, and helps timely repair.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of road inspection, and particularly relates to a road surface disease real-time inspection method and system based on a UAV. BACKGROUND

[0002] At present, road disease detection mainly relies on manual inspection or vehicle-mounted equipment. Manual inspection is low in efficiency, easily affected by subjective factors, and has safety hazards. Although vehicle-mounted equipment can improve efficiency, it is high in cost and poor in adaptability to complex road conditions. In addition, traditional methods are difficult to realize real-time detection and classification of diseases, cannot provide accurate disease information in time, and cause delay in maintenance decision-making.

[0003] Existing road inspection methods based on UAVs can improve detection efficiency, but still have deficiencies in image processing and disease identification. For example, existing methods have single technical means in image preprocessing, disease feature extraction and classification identification links, and cannot effectively handle complex lighting conditions, different terrains and multiple disease types.

[0004] In recent years, deep learning technology has made significant progress in image recognition and processing. However, when deep learning technology is applied to road disease detection, there are still many challenges. For example, how to realize efficient image processing and disease identification under the limited computing resources of UAVs, and how to ensure the accuracy and real-time performance of detection results, are problems to be solved at present.

[0005] Three-dimensional reconstruction technology has important application in road disease evaluation, but existing technologies have deficiencies in reconstruction accuracy and efficiency. For example, traditional three-dimensional reconstruction methods are prone to errors when dealing with complex disease morphology, and have high computational complexity, which is difficult to meet the needs of real-time inspection. SUMMARY

[0006] To solve the technical problems in the above background, the present application aims to solve the problems of low inspection efficiency, insufficient detection accuracy and inability to provide real-time feedback in the prior art.

[0007] To achieve the above purpose, the present application provides a road surface disease real-time inspection method based on a UAV, comprising the following steps:

[0008] Using a UAV to collect images of the road surface and perform preprocessing;

[0009] Using an improved YOLOv5 algorithm to detect disease targets in the preprocessed images;

[0010] Using an improved convolutional neural network to extract features of the detected disease targets;

[0011] The improved Vision Transformer algorithm is used to classify and identify the extracted disease features, and determine the type of disease.

[0012] After determining the type of disease, the severity of the disease is evaluated by a three-dimensional reconstruction algorithm, and a disease report is generated.

[0013] Preferably, the step of collecting images of the road surface includes: monitoring the light conditions in real time through a light intensity sensor, and adjusting the exposure parameters of the camera using an intelligent exposure algorithm, wherein the exposure parameters of the camera are adjusted according to the following formula:

[0014] ,

[0015] wherein E represents the exposure parameter of the camera, L represents the light intensity detected by the light intensity sensor, and a and b are determined coefficients for adjusting the exposure parameter according to different light conditions.

[0016] Preferably, the method for pre-processing the collected images includes: identifying and removing blurred images, overexposed images and underexposed images by calculating the gradient information and brightness distribution of the images; using a wavelet transform-based denoising algorithm to perform multi-scale denoising processing on the images; calculating the histogram of the images, generating a cumulative distribution function, and adjusting the pixel values of the images according to the cumulative distribution function; and applying a deep learning-based image super-resolution reconstruction algorithm to generate an adversarial network architecture.

[0017] Preferably, when using the improved YOLOv5 algorithm to detect disease targets, the channel attention mechanism is used to calculate the channel attention weight of the feature map:

[0018] ,

[0019] wherein, represents the channel attention weight of the feature map F, F is the feature map, is an activation function, MLP is a multi-layer perceptron, avgpool and maxpool are global average pooling and global maximum pooling respectively;

[0020] The spatial attention mechanism is introduced to calculate the spatial attention weight of the feature map:

[0021] ,

[0022] wherein, represents the spatial attention weight of the feature map F, conv is a convolution operation, and concat is a concatenation operation of the feature map.

[0023] The channel attention and spatial attention weights are applied to the feature map to obtain an enhanced feature map:

[0024] ,

[0025] wherein, represents the feature map after feature enhancement.

[0026] Preferably, the extracted disease features are classified by a hierarchical clustering algorithm, and the disease types include: cracks, pits, loose, rut, oil, spalling and settlement.

[0027] The disease features are dynamically learned using a meta-learning algorithm, and the specific formula is:

[0028] ,

[0029] wherein, represents the updated model parameters, represents the initial model parameters, is an adjustment parameter, represents the gradient of the model parameters , represents the task loss function, represents the model.

[0030] Preferably, the method for evaluating the severity of the disease by the three-dimensional reconstruction algorithm comprises:

[0031] ,

[0032] wherein, represents a point in a three-dimensional space, x is the horizontal coordinate on the image plane, y is the vertical coordinate on the image plane, z is the depth information of the point, f is the focal length of the camera, the depth value of the point in the depth map.

[0033] Preferably, the method for generating the disease report comprises: using wireless communication technology to transmit the detected disease information to the ground control center in real time; the ground control center receives and decodes the transmitted disease information, and processes the received disease information in real time, which includes screening, classification and sorting; the disease report includes disease type, location, size, severity, image segment of the disease area cropped from the original image, unique number of the disease area and detection timestamp; the generated disease report is stored in the database of the ground control center and backed up.

[0034] The application also provides a road surface disease real-time inspection system based on a UAV, which is used to realize the above method and comprises an acquisition module, a detection module, an extraction module, an identification module and a generation module.

[0035] The collection module is used for image collection of the road surface by the unmanned aerial vehicle and preprocessing;

[0036] The detection module is used for disease target detection on the preprocessed image by using an improved YOLOv5 algorithm;

[0037] The extraction module is used for feature extraction of the detected disease target by using an improved convolutional neural network;

[0038] The recognition module is used for classification and recognition of the extracted disease features by using an improved Vision Transformer algorithm, so as to determine the type of the disease;

[0039] The generation module is used for evaluating the severity of the disease and generating a disease report by a three-dimensional reconstruction algorithm after the type of the disease is determined.

[0040] Preferably, the working procedure of image collection of the road surface comprises the following steps: real-time monitoring of light conditions by a light intensity sensor, and adjusting the exposure parameter of a camera by using an intelligent exposure algorithm, wherein the exposure parameter of the camera is adjusted according to the following formula:

[0041]

[0042] wherein E represents the exposure parameter of the camera, L represents the light intensity detected by the light intensity sensor, and a and b are determined coefficients for adjusting the exposure parameter according to different light conditions.

[0043] Preferably, the procedure of image preprocessing comprises the following steps: identifying and removing blurred images, overexposed images and underexposed images by calculating the gradient information and brightness distribution of the image; performing multi-scale denoising processing on the image by using a denoising algorithm based on wavelet transform; calculating the histogram of the image, generating a cumulative distribution function, and adjusting the pixel value of the image according to the cumulative distribution function; and applying an image super-resolution reconstruction algorithm based on deep learning to generate a generative adversarial network architecture.

[0044] Compared with the prior art, the present application has the following beneficial effects:

[0045] The present application can significantly improve the efficiency of road disease inspection by automatically collecting images by the unmanned aerial vehicle, combining efficient image processing and disease identification algorithms, and can cover a large area in a short time, reducing the labor cost and time cost.

[0046] The present application can accurately detect and identify various types of diseases by using the improved YOLOv5 algorithm and Vision Transformer algorithm, improve the accuracy of disease detection, and the application of multi-scale feature extraction and deep learning technology makes the disease features more obvious, further improving the accuracy of detection.​

[0047] The present application realizes real-time transmission and processing of disease information, and the ground control center can obtain disease information in time and generate a disease report, so that the road maintenance department can quickly respond to the disease and take timely repair measures, thereby improving the efficiency and response speed of road maintenance.

[0048] The present application can not only measure the size of the disease, but also reconstruct the three-dimensional morphology of the disease through three-dimensional reconstruction technology, comprehensively evaluate the severity of the disease, and provide more comprehensive disease information for the road maintenance department, which is helpful for formulating a more reasonable maintenance plan. BRIEF DESCRIPTION OF DRAWINGS

[0049] In order to more clearly illustrate the technical solutions of the present application, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings described in the following embodiments are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0050] Figure 1 The present application is a method flowchart of the embodiment;

[0051] Figure 2 The present application is a principle and process diagram of convolution of the embodiment; wherein (a) represents a convolution principle diagram; (b) represents a convolution process diagram;

[0052] Figure 3 The present application is a purpose and principle diagram of pooling of the embodiment; wherein (a) represents a pooling principle diagram; (b) represents a pooling purpose diagram. DETAILED DESCRIPTION

[0053] The technical solutions in the embodiments of the present application will be described in detail below with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0054] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0055] Embodiment one:

[0056] As shown in the method flowchart of the embodiment, the steps include: Figure 1

[0057] S1. An unmanned aerial vehicle is used to collect images of the road surface and perform preprocessing.​

[0058] The unmanned aerial vehicle equipped with a camera and a positioning system is used to collect images of the road surface according to a preset flight path. Before takeoff, the camera of the unmanned aerial vehicle is calibrated to detect the imaging quality of the camera through a multi-point light intensity detection and color correction algorithm to reduce image distortion; according to the terrain and environmental conditions of the road, the height and speed of the preset flight path are adjusted, and the flight path of the unmanned aerial vehicle is optimized through the terrain matching algorithm and environmental perception function of the positioning system to realize complete collection of images.

[0059] During the image collection process, the light intensity sensor is used to monitor the lighting conditions in real time, and the intelligent exposure algorithm is used to adjust the exposure parameters of the camera to improve the clarity of the images. The exposure parameters of the camera are adjusted according to the following formula:

[0060] ,

[0061] where E represents the exposure parameter of the camera, L represents the light intensity detected by the light intensity sensor, and a and b are determined coefficients for adjusting the exposure parameter according to different lighting conditions.

[0062] After the image collection is completed, the collected images are screened, and blurred images, overexposed images and underexposed images are identified and removed by calculating the gradient information and brightness distribution of the images. A wavelet transform-based denoising algorithm is used to perform multi-scale denoising processing on the images, and an adaptive threshold selection algorithm is used to retain the edge information of the images while removing noise interference. The histogram of the image is calculated to generate a cumulative distribution function, and the pixel values of the image are adjusted according to the cumulative distribution function to enhance the contrast of the image; a deep learning-based image super-resolution reconstruction algorithm is applied to improve the resolution of the image and enhance the detail information through a generative adversarial network architecture.

[0063] S2. Use the improved YOLOv5 algorithm to detect disease targets in the preprocessed images.

[0064] The background of the preprocessed image is segmented, and a Gaussian mixture model is used to model the background pixels, and the matching degree of each pixel point with the background model is calculated, and the specific formula is:

[0065] ,

[0066] where, is the matching degree of pixel point p with the background model, p represents a pixel point in the image, exp represents the exponential operation of the natural exponential function e, is the intensity value of pixel point p, and are the mean and standard deviation of the background model.

[0067] According to the matching degree, the pixel points are classified into background regions and foreground regions, and the foreground regions are extracted as possible disease regions. The improved YOLOv5 algorithm is used for disease target detection, and a channel attention mechanism is introduced to calculate the channel attention weight of the feature map, and the specific formula is:

[0068] ,

[0069] wherein, represents the channel attention weight of the feature map F, F is the feature map, is an activation function, MLP is a multi-layer perceptron, avgpool and maxpool are global average pooling and global maximum pooling respectively.

[0070] A spatial attention mechanism is introduced to calculate the spatial attention weight of the feature map, and the specific formula is:

[0071] ,

[0072] wherein, represents the spatial attention weight of the feature map F, conv is a convolution operation, and concat is a feature map splicing operation.

[0073] The channel attention and spatial attention weights are applied to the feature map to enhance the feature representation of the key region, and the specific formula is:

[0074] ,

[0075] wherein, represents the feature map after feature enhancement.

[0076] The improved YOLOv5 algorithm is used for target detection on the enhanced feature map, and the position, range and confidence of the disease target are output.

[0077] S3. An improved convolutional neural network is used to extract features of the detected disease target.

[0078] Different size convolution kernels are used to perform convolution operations on the feature map of the disease target to extract multi-scale features. The multi-scale feature maps are fused to generate a comprehensive feature map. Combined with the Transformer architecture, the features are analyzed through the self-attention mechanism and the multi-head attention module to extract long-distance dependency relationships. Then the extracted features are normalized through batch normalization and layer normalization techniques to adjust the mean and variance of the feature map, optimize the distribution of the features, and realize the consistency of the features.

[0079] The improved convolutional neural network described above is composed of convolutional layers and pooling layers, and the specific structure includes:

[0080] AsFigure 2 (a), convolution refers to using a batch of filters to move on the picture to collect small block area information, here the filter is called a convolution kernel, the number of filters is called the number of convolution kernels, the size of the filter is called the size of the convolution kernel, and the movement of the filter is divided into horizontal and vertical directions, and the interval of the movement is called the stride. The original size contains three parameters of length, width and height, where height refers to 3 for color pictures and 1 for black and white pictures, and after filtering and merging, higher layer information is obtained, where "filtering and merging" refers to one convolution layer, and "obtaining higher layer information" refers to that the output information is continuously compressed in length and width while increasing in height (height refers to the number of channels), such as Figure 2 (b), obtaining information with a deeper understanding of the picture information.

[0081] For example, Figure 3 (a), pooling refers to that every convolution will cause unconscious information loss, in order to solve this problem, a pooling layer is added after each convolution layer to filter data and compress length and width. The pooling layer ensures the integrity of the convolution layer information and the effective filtering of the input information, reduces the burden of network calculation, and effectively improves the accuracy of the model. For example, Figure 3 (b), important parameters in the pooling layer include the size of the pooling kernel function (k size ) mxn, that is, each m-length and n-wide region is pooled once; pooling types such as maximum pooling and average pooling, such as maximum pooling, which takes the maximum value of each pooled region as a representation; and stride, which refers to the interval size of the movement of the pooling kernel function. The convolution and pooling layer also has a parameter of padding, which refers to the processing method of the picture boundary, such as selecting padding as SAME, which means that the convolution layer input and output can maintain the same size by filling 0 in the insufficient part. For example, Figure 3 (b) is a pooling layer with a pooling kernel function size of 2x2 and a stride of 2x2 (2 in horizontal and vertical directions), and the pooling method is maximum pooling. It can be seen that the left 4x4 matrix becomes a 2x2 matrix after pooling, and each value after pooling is the maximum value of the corresponding pooling region in the original matrix.

[0082] S4. Using the improved Vision Transformer algorithm to classify and identify the extracted disease features to determine the type of disease.

[0083] Classifying the extracted disease features and identifying the disease type through a hierarchical clustering algorithm, wherein the disease type includes cracks, pits, loose, rut, oil bleeding, spalling and settlement.

[0084] Initialize the parameters of the meta-learning algorithm, and use the meta-learning algorithm to dynamically learn the disease characteristics and adapt to the disease types, with the specific formula being:

[0085] ,

[0086] wherein, represents the updated model parameters, represents the initial model parameters, is an adjustment parameter, represents the gradient of the model parameters , represents the task loss function, represents the model, and the parameters of .

[0087] Use the dynamic feature extraction module to adjust the feature extraction process according to the distribution of disease characteristics, with the specific formula being:

[0088] ,

[0089] wherein, represents the feature vector after dynamic feature extraction, Feature Extractor represents the feature extraction module, which is a neural network module used to extract useful features from input features, and T represents the input feature vector, represents the updated model parameters.

[0090] Use the dynamic weight adjustment mechanism to dynamically adjust the weights according to the distribution of disease characteristics, with the specific formula being:

[0091] ,

[0092] wherein, represents the updated model parameters, represents the initial model parameters, is an adjustment parameter, represents the adjustment amount of the parameters.

[0093] Use the adaptive classifier adjustment algorithm to adjust the weights of the classifier according to the distribution of disease characteristics, with the specific formula being:

[0094] ,

[0095] ,

[0096] wherein, represents the weight of the i-th disease, exp is the exponential operation of the natural exponential function e, is an adjustment parameter used to control the degree of influence of confidence on weights, Indicates disease characteristics The confidence score, This represents the extracted disease features, n represents the total number of disease categories, j is an index variable used to iterate through all disease categories, and Classifier is a classifier function used to classify the disease features.

[0097] S5. After the disease type is determined, the severity of the disease is assessed using a three-dimensional reconstruction algorithm and a disease report is generated.

[0098] A 3D reconstruction algorithm based on deep learning is applied to generate a depth map of the diseased area from the image, wherein the depth map represents the depth information of each pixel.

[0099] Point cloud data of the diseased area is generated based on the depth map. Using the depth information in the depth map and combined with camera parameters, the 3D coordinates of each pixel are calculated. The specific formula is as follows:

[0100] ,

[0101] in, This represents a point in three-dimensional space, where x is the horizontal coordinate on the image plane, y is the vertical coordinate on the image plane, z is the depth information of the point, and f is the camera's focal length. Points in depth map The depth value.

[0102] Point cloud data is converted into voxel meshes, and isosurfaces are extracted from the voxel meshes using the Marching Cubes algorithm to generate a 3D model of the disease. The reconstructed 3D model is analyzed, and the geometric features of the diseased area are measured using volume calculation and shape analysis algorithms to assess the severity of the disease. The geometric features include volume, aspect ratio, and curvature, and the severity of the disease is categorized as mild, moderate, and severe.

[0103] Finally, wireless communication technology is used to transmit the detected disease information to the ground control center in real time. The ground control center receives and decodes the transmitted disease information and processes it in real time, including filtering, classifying, and sorting.

[0104] Based on the processing results, a disease report is generated, which includes the disease type, location, size, severity, image fragments of the diseased area cropped from the original image, a unique number of the diseased area, and a detection timestamp. The generated disease report is stored in the database of the ground control center and backed up.

[0105] Example 2:

[0106] The embodiment also provides a UAV-based real-time road surface disease inspection system, which comprises a collection module, a detection module, an extraction module, an identification module and a generation module.

[0107] The embodiment will be described below in detail to illustrate how the present application solves the technical problems in actual life.

[0108] First, the collection module collects images of the road surface by using a UAV and performs preprocessing.

[0109] The UAV equipped with a camera and a positioning system collects images of the road surface according to a preset flight route. Before taking off, the camera of the UAV is calibrated, the imaging quality of the camera is detected through a multi-point light intensity detection and color correction algorithm, and image distortion is reduced; according to the terrain and environmental conditions of the road, the height and speed of the preset flight route are adjusted, the flight path of the UAV is optimized through a terrain matching algorithm and an environmental perception function of the positioning system, and complete image collection is realized.

[0110] During the image collection process, the light intensity sensor is used to monitor the light conditions in real time, and the intelligent exposure algorithm is used to adjust the exposure parameters of the camera to improve the clarity of the images. The exposure parameters of the camera are adjusted according to the following formula:

[0111] ,

[0112] Wherein, E represents the exposure parameter of the camera, L represents the light intensity detected by the light intensity sensor, and a and b are determined coefficients for adjusting the exposure parameter according to different light conditions.

[0113] After the image collection is completed, the collected images are screened, blurred images, overexposed images and underexposed images are identified and removed by calculating the gradient information and brightness distribution of the images. A denoising algorithm based on wavelet transform is used to perform multi-scale denoising processing on the images, and an adaptive threshold selection algorithm is used to retain the edge information of the images while removing noise interference. The histogram of the image is calculated, the cumulative distribution function is generated, and the pixel value of the image is adjusted according to the cumulative distribution function to enhance the contrast of the image; a deep learning-based image super-resolution reconstruction algorithm is applied, a generative adversarial network architecture is used to improve the resolution of the image and enhance the detail information.

[0114] The detection module uses an improved YOLOv5 algorithm to detect disease targets in the preprocessed images.

[0115] The preprocessed images are segmented, the background pixels are modeled using a Gaussian mixture model, and the matching degree of each pixel point with the background model is calculated, and the specific formula is as follows:

[0116] ,

[0117] wherein, is the matching degree of the pixel point p background model, p represents a pixel point in the image, exp represents the exponential operation of the natural exponential function e, is the intensity value of the pixel point p, and is the mean and standard deviation of the background model.

[0118] According to the matching degree, the pixel points are classified into background regions and foreground regions, and the foreground regions are extracted as possible disease regions. The improved YOLOv5 algorithm is used for disease target detection, and a channel attention mechanism is introduced to calculate the channel attention weight of the feature map, and the specific formula is:

[0119] ,

[0120] wherein, represents the channel attention weight of the feature map F, F is the feature map, is an activation function, MLP is a multi-layer perceptron, avgpool and maxpool are global average pooling and global maximum pooling respectively.

[0121] A spatial attention mechanism is introduced to calculate the spatial attention weight of the feature map, and the specific formula is:

[0122] ,

[0123] wherein, represents the spatial attention weight of the feature map F, conv is a convolution operation, and concat is a feature map splicing operation.

[0124] The channel attention and spatial attention weights are applied to the feature map to enhance the feature representation of the key region, and the specific formula is:

[0125] ,

[0126] wherein, represents the feature map after feature enhancement.

[0127] The improved YOLOv5 algorithm is used for target detection on the enhanced feature map, and the position, range and confidence of the disease target are output.

[0128] The improved convolutional neural network is used for feature extraction of the detected disease target.

[0129] The improved convolutional neural network is composed of convolutional layers and pooling layers, and the specific structure includes:

[0130] As Figure 2(a), convolution refers to using a batch of filters to move on the picture to collect small block area information, here the filter is called the convolution kernel, the number of filters is called the number of convolution kernels, the size of the filter is called the size of the convolution kernel, and the movement of the filter is divided into horizontal and vertical directions, and the interval of the movement is called the stride. The original size contains three parameters of length, width and height, where height refers to 3 for color pictures and 1 for black and white pictures. After filtering and merging, higher level information is obtained. Here "filtering and merging" refers to one convolution layer, and "obtaining higher level information" refers to that the output information is constantly compressed in length and width while increasing in height (height refers to the number of channels). For example Figure 2 (b), the information obtained is more in-depth understanding of the picture information.

[0131] For example Figure 3 (a), pooling refers to the fact that each convolution will cause unintentional information loss. In order to solve this problem, the length and width of the image are not compressed but only the height is increased during the convolution process, and a pooling layer is added after each convolution layer to filter data and compress the length and width. The pooling layer ensures the integrity of the convolution layer information and the effective filtering of the input information, reduces the burden of network calculation, and effectively improves the accuracy of the model. For example Figure 3 (b), the important parameters in the pooling layer include the size of the pooling kernel function (k size ) m x n, which means that each m x n region is pooled once; the pooling type is maximum pooling, average pooling, etc. For example, maximum pooling refers to taking the maximum value of the gray scale of each pooling region as the representation; the stride is the interval size of the movement of the pooling kernel function. Convolution and pooling layer also have a parameter called padding, which refers to the processing method of the picture boundary. For example, if the padding is SAME, it means that the convolution layer input and output can maintain the same size by filling 0 in the insufficient part. For example Figure 3 (b) is a pooling layer with a pooling kernel function size of 2 x 2 and a stride of 2 x 2 (2 in both horizontal and vertical directions). The pooling method is maximum pooling. As can be seen, the left 4 x 4 matrix becomes a 2 x 2 matrix after pooling, and each value after pooling is the maximum value of the corresponding pooling region in the original matrix.

[0132] Different size convolution kernels are used to perform convolution operations on the feature map of the disease target to extract multi-scale features. The multi-scale feature maps are fused to generate a comprehensive feature map. Combined with the Transformer architecture, the features are analyzed through the self-attention mechanism and the multi-head attention module to extract long-distance dependencies. Then the extracted features are normalized by batch normalization and layer normalization techniques to adjust the mean and variance of the feature map and optimize the distribution of the features to achieve consistency of the features.

[0133] The recognition module uses an improved Vision Transformer algorithm to classify and recognize the extracted disease features, and determines the type of disease.

[0134] The extracted disease features are classified, and the disease types are identified through a hierarchical clustering algorithm, including cracks, pits, loose, rut, oil, spalling and settlement.

[0135] The parameters of the meta-learning algorithm are initialized, and the meta-learning algorithm is used to dynamically learn the disease features and adapt to the disease types, with the specific formula being:

[0136] ,

[0137] wherein, represents the updated model parameters, represents the initial model parameters, is an adjustment parameter, represents the gradient of the model parameters , and represents the task loss function, represents the model, and the parameters thereof are .

[0138] The dynamic feature extraction module is used to adjust the feature extraction process according to the distribution of disease features, with the specific formula being:

[0139] ,

[0140] wherein, represents the feature vector after dynamic feature extraction, Feature Extractor represents the feature extraction module, which is a neural network module used to extract useful features from input features, and T represents the input feature vector, represents the updated model parameters.

[0141] The dynamic weight adjustment mechanism is used to dynamically adjust the weights according to the distribution of disease features, with the specific formula being:

[0142] ,

[0143] wherein, represents the updated model parameters, represents the initial model parameters, is an adjustment parameter, represents the adjustment amount of the parameters.

[0144] The adaptive classifier adjustment algorithm is used to adjust the weights of the classifier according to the distribution of disease features, with the specific formula being:

[0145] ,

[0146] ,

[0147] wherein, wi represents the weight of the ith disease, exp is the exponential operation of the natural exponential function e, is an adjustment parameter, used to control the degree of influence of the confidence on the weight, represents the confidence score of the disease feature , represents the extracted disease feature, n represents the total number of disease categories, j is an index variable for traversing all disease categories, and Classifier is a classifier function for classifying disease features.

[0148] The final generation module, after determining the disease type, evaluates the severity of the disease and generates a disease report through a three-dimensional reconstruction algorithm.

[0149] A three-dimensional reconstruction algorithm based on deep learning is applied to generate a depth map of the disease area from the image, which represents the depth information of each pixel point.

[0150] Point cloud data of the disease area is generated according to the depth map. Using the depth information in the depth map, combined with the camera parameters, the three-dimensional coordinates of each pixel point are calculated, and the specific formula is:

[0151] ,

[0152] wherein, represents a point in three-dimensional space, x is the horizontal coordinate on the image plane, y is the vertical coordinate on the image plane, z is the depth information of the point, f is the focal length of the camera, the depth value of the point in the depth map.

[0153] The point cloud data is converted into a voxel grid, and the Marching Cubes algorithm is used to extract isosurfaces from the voxel grid to generate a three-dimensional model of the disease. The reconstructed three-dimensional model is analyzed, and the geometric features of the disease area are measured through volume calculation and shape analysis algorithms to evaluate the severity of the disease, including volume, aspect ratio and curvature, and the severity of the disease includes mild, moderate and severe.

[0154] Finally, the detected disease information is transmitted in real time to the ground control center using wireless communication technology. The ground control center receives and decodes the transmitted disease information, and processes the received disease information in real time, including screening, classification and sorting.

[0155] Based on the processing results, a disease report is generated, which includes the disease type, location, size, severity, image fragments of the diseased area cropped from the original image, a unique number of the diseased area, and a detection timestamp. The generated disease report is stored in the database of the ground control center and backed up.

[0156] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for real-time inspection of road surface defects based on unmanned aerial vehicles (UAVs), characterized by the following steps: include: UAVs were used to collect and preprocess images of the road surface; The improved YOLOv5 algorithm is used to detect disease targets in the preprocessed image; the improved YOLOv5 algorithm calculates the channel attention weights of the feature map through an ingress channel attention mechanism: ， in, This represents the channel attention weights of the feature map F, where F is the feature map. It is the activation function, MLP is the multilayer perceptron, and avgpool and maxpool are global average pooling and global max pooling, respectively. Introducing a spatial attention mechanism to calculate the spatial attention weights of the feature map: ， in, This represents the spatial attention weights of feature map F, where conv is the convolution operation and concat is the feature map concatenation operation. By applying channel attention and spatial attention weights to the feature map, we obtain the enhanced feature map: ， in, This represents the feature map after feature enhancement. An improved convolutional neural network is used to extract features from the detected disease targets; the improved convolutional neural network consists of convolutional layers and pooling layers, wherein the pooling kernel function of the pooling layer has a size of 2×2, a stride of 2×2, and the pooling method is max pooling; Convolutional operations are performed on the feature maps of disease targets using convolutional kernels of different scales to extract multi-scale features. The multi-scale feature maps are then fused to generate a comprehensive feature map. Combined with the Transformer architecture, the features are analyzed through a self-attention mechanism and a multi-head attention module to extract long-distance dependencies. Subsequently, the extracted features are normalized by batch normalization and layer normalization techniques to adjust the mean and variance of the feature maps and optimize the feature distribution. The extracted disease features are classified and identified using an improved Vision Transformer algorithm to determine the type of disease. The model parameters of the improved Vision Transformer algorithm are obtained using the following formula: ， in, This represents the updated model parameters. This represents the initial model parameters. It's about adjusting parameters. Indicates the model parameters gradient, This represents the task loss function. The model is represented by the following parameters: ; The improved Vision Transformer algorithm includes a dynamic feature extraction module, a dynamic weight adjustment mechanism, and an adaptive classifier adjustment algorithm. The dynamic feature extraction module adjusts the feature extraction process based on the distribution of disease features. The specific formula is as follows: ， in, This represents the feature vector after dynamic feature extraction. The Feature Extractor is a neural network module used to extract useful features from the input features. T represents the input feature vector. This represents the updated model parameters; A dynamic weight adjustment mechanism is used, which dynamically weights the disease characteristics based on their distribution. The specific formula is as follows: ， in, This represents the updated model parameters. This represents the initial model parameters. It's about adjusting parameters. Indicates the amount of parameter adjustment; An adaptive classifier adjustment algorithm is used, which adjusts the classifier weights based on the distribution of disease characteristics. The specific formula is as follows: ， ， in, Let represent the weight of the i-th type of disease, and exp be the exponentiation of the natural exponential function e. These are parameters that are adjusted to control the degree to which the confidence level affects the weights. Indicates disease characteristics The confidence score, This represents the extracted disease features, where n represents the total number of disease categories, j is an index variable used to iterate through all disease categories, and Classifier is a classifier function used to classify the disease features. Once the disease type is determined, the severity of the disease is assessed using a 3D reconstruction algorithm, and a disease report is generated.

2. The method for real-time inspection of road surface defects based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The steps for acquiring images of the road surface include: real-time monitoring of lighting conditions using a light intensity sensor, and adjusting the camera's exposure parameters using an intelligent exposure algorithm. The camera's exposure parameters are adjusted according to the following formula: ， Where E represents the camera's exposure parameters, L represents the light intensity detected by the light intensity sensor, and a and b are fixed coefficients used to adjust the exposure parameters according to different lighting conditions.

3. The method for real-time inspection of road surface defects based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The methods for preprocessing the acquired images include: identifying and removing blurred, overexposed, and underexposed images by calculating the gradient information and brightness distribution of the images; performing multi-scale denoising on the images using a wavelet transform-based denoising algorithm; calculating the image histogram, generating a cumulative distribution function, and adjusting the pixel values ​​of the images according to the cumulative distribution function; and applying a deep learning-based image super-resolution reconstruction algorithm to generate an adversarial network architecture.

4. The method for real-time inspection of road surface defects based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The extracted disease features are classified using a hierarchical clustering algorithm. The disease types include: cracks, potholes, loosening, rutting, oil seepage, spalling, and settlement. The meta-learning algorithm is used to dynamically learn the disease features. The specific formula is as follows: ， in, This represents the updated model parameters. This represents the initial model parameters. It's about adjusting parameters. Indicates the model parameters gradient, This represents the task loss function. Representation model.

5. The method for real-time inspection of road surface defects based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, Methods for assessing the severity of diseases using 3D reconstruction algorithms include: ， in, This represents a point in three-dimensional space, where x is the horizontal coordinate on the image plane, y is the vertical coordinate on the image plane, z is the depth information of the point, and f is the camera's focal length. Points in depth map The depth value.

6. The method for real-time inspection of road surface defects based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The method for generating the disease report includes: using wireless communication technology to transmit the detected disease information to the ground control center in real time; the ground control center receiving and decoding the transmitted disease information, and processing the received disease information in real time, the processing including filtering, classification and sorting; the disease report including disease type, location, size, severity, image fragments of the disease area cropped from the original image, unique number of the disease area and detection timestamp; storing the generated disease report in the database of the ground control center and backing it up.

7. A real-time road surface defect inspection system based on unmanned aerial vehicles (UAVs), the system being used to implement the method described in any one of claims 1-6, characterized in that, include: The module includes a data acquisition module, a detection module, an extraction module, a recognition module, and a generation module. The acquisition module is used to acquire and preprocess images of the road surface using a drone. The detection module is used to detect disease targets in the preprocessed image using the improved YOLOv5 algorithm; The extraction module shown is used to extract features from the detected disease targets using an improved convolutional neural network; The identification module is used to classify and identify the extracted disease features using the improved Vision Transformer algorithm to determine the type of disease; The generation module is used to assess the severity of the disease and generate a disease report after the disease type is determined, through a three-dimensional reconstruction algorithm.

8. The real-time road surface defect inspection system based on unmanned aerial vehicles (UAVs) according to claim 7, characterized in that, The workflow for acquiring images of the road surface includes: real-time monitoring of lighting conditions using a light intensity sensor, and adjusting the camera's exposure parameters using an intelligent exposure algorithm. The camera's exposure parameters are adjusted according to the following formula: ， Where E represents the camera's exposure parameters, L represents the light intensity detected by the light intensity sensor, and a and b are fixed coefficients used to adjust the exposure parameters according to different lighting conditions.

9. The real-time road surface defect inspection system based on unmanned aerial vehicles (UAVs) according to claim 7, characterized in that, The image preprocessing process includes: identifying and removing blurred, overexposed, and underexposed images by calculating the gradient information and brightness distribution of the image; performing multi-scale denoising on the image using a wavelet transform-based denoising algorithm; calculating the image histogram, generating a cumulative distribution function, and adjusting the image pixel values ​​according to the cumulative distribution function; and applying a deep learning-based image super-resolution reconstruction algorithm to generate an adversarial network architecture.

Images