Road detection device and method based on unmanned aerial vehicle and PDA (Personal Digital Assistant) road map system

Abstract

The invention relates to the technical field of road detection, and discloses a road detection device and method based on an unmanned aerial vehicle and a PDA road map system, and the device comprises a control providing module which is used for importing an electronic map into the PDA road map system, matching the flight path of the unmanned aerial vehicle with the PDA road map system, and planning the flight path of the unmanned aerial vehicle; the unmanned aerial vehicle acquisition module is used for acquiring a road surface image according to the flight path by using an unmanned aerial vehicle carrying multiple sensors; the communication processing module is used for carrying out real-time data transmission on various data and carrying out interaction operation; and the data processing module is used for carrying out restoration processing on the pavement image based on the variation model. A flight path of an unmanned aerial vehicle along a road is planned, an unmanned aerial vehicle acquisition module synchronously triggers multiple sensors to acquire multi-modal road surface data, various data are transmitted in real time, a degraded image in a complex environment is restored based on a variational model, and road surface defects are analyzed through a deep convolutional neural network fused with physical prior.

Description

Technical Field

[0001] This invention relates to the field of highway inspection technology, specifically to a highway inspection device and method based on a drone and a PDA road map system. Background Technology

[0002] As a core component of road maintenance throughout its entire lifecycle, highway inspection requires the accurate identification of defects such as cracks and potholes and their spatial location to provide a basis for maintenance decisions.

[0003] Current road inspection technology mainly uses manual inspection combined with portable devices (such as PDAs) and map software for road inspection. However, this method is highly dependent on manpower, greatly limited by environmental factors, and difficult to achieve comprehensive coverage and real-time monitoring, resulting in low inspection efficiency. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a highway inspection device and method based on a drone and a PDA road map system, which solves the problem of low efficiency in traditional highway inspection.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a highway detection device based on an unmanned aerial vehicle (UAV) and a PDA road map system, comprising: Control Module: Used to import electronic maps into the PDA road map system, match the drone's flight trajectory with the PDA road map system, and plan the drone's flight path; Drone data acquisition module: Used to acquire road surface images based on the flight path of a drone equipped with multiple sensors; Communication processing module: used for real-time data transmission and interactive operations; Data processing module: used to restore road surface images based on variational models, eliminate the effects of image degradation in complex environments, and obtain processed data; Intelligent recognition module: used to analyze the processed data and identify road surface defects by fusing physical priors into a deep convolutional neural network; Matching output module: Used to match road surface defects with coordinates in the PDA road map system to generate an inspection report containing defect information.

[0006] By adopting the above technical solution, the drone's flight path along the highway is planned, and the drone's data acquisition module synchronously triggers multiple sensors to collect multimodal road surface data. The data is transmitted in real time, and the degraded image under complex environment is restored based on the variational model. The road surface defects are analyzed by a deep convolutional neural network that integrates physical priors, and an inspection report is generated that is associated with the coordinates of the PDA road map system. This constructs a fully automated closed-loop inspection process, replacing the traditional manual inspection mode, shortening the time loss of inspection per kilometer of highway, and solving the problem of low inspection efficiency in traditional highway inspection.

[0007] Preferably, the control providing module includes: Electronic map import unit: used to import electronic maps into the PDA road map system, generate a highway network topology structure including road centerlines, curve curvature and road surface material distribution. The nodes of the highway network topology structure are road intersections, the edges are road segments, and it is accompanied by geometric coordinates and attribute labels. Trajectory matching unit: Used to perform coordinate matching between the preset flight trajectory of the UAV and the centerline of the road based on the road network topology, generating a set of geotagged path points, wherein the coordinate matching satisfies X max =T(X) drone ), where X max For electronic map coordinates, X drone Here are the UAV's GPS coordinates, and T(·) is the affine transformation function whose parameters are pre-calibrated by the PDA roadmap system; Parameter calculation unit: Used to calculate UAV flight parameters based on a path point set and preset detection specifications, and generate a flight path containing 3D path instructions. The detection specifications include flight altitude range, flight speed range, camera exposure time, and image acquisition interval. The length of the motion blur kernel in the calculation of UAV flight parameters... Where v is the drone's flight speed, Δt is the camera exposure time, s is the camera pixel size, and h is the flight altitude.

[0008] Preferably, the UAV data acquisition module includes: Multi-sensor control and acquisition unit: used to synchronously trigger multiple sensors according to the flight path to align data timestamps. The multiple sensors include an RGB camera, a multispectral sensor, and a GPS / IMU. Parameter synchronization unit: Used to collect road surface data including visible light images of the road surface, reflectivity data, real-time positioning data and attitude data using multiple sensors, and perform spatial registration to generate a road surface image with geographic coordinates.

[0009] Preferably, the communication processing module includes: Data Packaging Unit: Used to divide various data items into frames, add timestamps and check codes, and generate a data frame sequence; Dual-channel switching unit: used to monitor the signal strength of the 4G / 5G main channel in real time, and switch to the Bluetooth backup channel when the signal is below the threshold for data transmission.

[0010] Preferably, the data processing module includes: Degradation parameter estimation unit: used to calculate the atmospheric scattering coefficient of road surface images based on multispectral data, and to calculate the motion blur kernel of road surface images based on GPS speed and camera parameters; Variational model building unit: used to build a variational model containing data fidelity terms, TV regularization terms, and nonlocal mean terms based on atmospheric scattering coefficients and motion fuzzy kernels; ADMM solver unit: used to iteratively solve variational models using the alternating direction multiplier method to generate non-degenerate processed data.

[0011] Preferably, the atmospheric scattering coefficient is Where t is the average transmittance and h is the flight altitude; The motion blur kernel is in, Let v be the length of the motion blur kernel, v be the real-time GPS speed of the UAV, Δt be the camera exposure time, s be the camera pixel size, and rect(·) be the rectangular function that satisfies... u represents the pixel displacement; The variational model is: Where J(x,y) is the restored image, A is the atmospheric light component, and k(u) is the motion blur kernel. rect(·) is a rectangle function. Let N(x,y) be the pixel block, λ and γ be the regularization coefficients, t be the mean transmittance, and w be the similarity weight. ′ ,y ′ ) represents the pixel coordinates paired with (x,y) in the nonlocal mean term; The alternating direction multiplier method is as follows: U k+1 =U k +J k+1 +Z k+1 ; The convergence condition is ||J k+1 -J k ||2<10 -4 , where J k Z is the restored image for the k-th iteration. k Auxiliary variable, U k Let ρ be the dual variable and ρ be the penalty parameter. The proximal operator for the TV regularization term.

[0012] Preferably, the intelligent recognition module includes: Physical feature extraction unit: used to enhance crack direction features in the processed data through Gabor filtering, and to filter out false positive regions in the processed data through spectral constraints, thereby extracting physical feature data that incorporates physical priors; Semantic Analysis Unit: Used to capture the continuity of cracks in the processed data through a lightweight Transformer to generate a semantic feature map of defects; Fusion Decision Unit: Used to fuse physical feature data and defect semantic feature map through attention mechanism, and optimize based on multi-scale loss function in deep convolutional neural network to output road surface defect, which includes type, location, features and confidence level.

[0013] Preferably, the Gabor filter kernel formula is as follows: exp(i2πfx ′ ), where (x,y) are the pixel coordinates of the processed data, θ is the principal direction of the crack, f is the spatial frequency, and σ x σ y Let be the Gaussian kernel standard deviation, exp(i2πfx) ′ ) represents a complex exponential term; The attention formula for the lightweight Transformer is as follows: Where Q, K, and V represent the query, key, and value projection of the physical feature data, respectively. T To calculate the similarity between features, The scaling factor, softmax(·) is the normalization function; The attention mechanism is F fusion =Attention(F phys F sem F sem ), where F phys For physical characteristic data, F sem For the defect semantic feature map, F fusion To integrate physical feature data with features associated with the defect semantic feature map; The multi-scale loss function is: Where α, β, and γ are the loss weights. For Focal Loss, For physical constraint loss, This is a bi-branch feature consistency loss.

[0014] Preferably, the matching output module includes: Coordinate transformation unit: used to map the image pixel coordinates of road surface defects to the latitude and longitude of the electronic map through projection transformation algorithm, and generate defect locations with geographic labels; Report generation unit: Used to classify defect locations according to defect characteristics and generate structured reports that include location, type, size, and severity; Visualization output unit: used to render a defect heatmap in the PDA roadmap system based on the structured report, display the defect level through color coding, and generate an inspection report containing defect information.

[0015] The highway detection method based on UAV and PDA road map system, applied to the aforementioned highway detection device based on UAV and PDA road map system, includes the following steps: Control provision: Import electronic maps into the PDA road map system, match the drone's flight trajectory with the PDA road map system, and plan the drone's flight path; Drone data collection: Using drones equipped with multiple sensors to collect road surface images based on their flight paths; Communication processing: Real-time transmission of various data and interactive operation; Data processing: The road surface image is restored based on the variational model to eliminate the image degradation effect under complex environment and obtain the processed data; Intelligent recognition: By analyzing the processed data through a deep convolutional neural network that integrates physical priors, road surface defects are identified; Matching output: The road surface defects are matched with the coordinates in the PDA road map system to generate a detection report containing defect information.

[0016] This invention provides a highway detection device and method based on an unmanned aerial vehicle (UAV) and a PDA road map system. It has the following beneficial effects: 1. In this invention, by planning the flight path of the UAV along the highway, the UAV acquisition module synchronously triggers multiple sensors to collect multimodal road surface data, transmits the data in real time, and restores the degraded image under complex environment based on the variational model. By analyzing road surface defects through a deep convolutional neural network that integrates physical priors, an inspection report associated with the coordinates of the PDA road map system is generated, and a fully automated closed-loop inspection process is constructed to replace the traditional manual inspection mode, shorten the time loss of inspection per kilometer of highway, and solve the problem of low inspection efficiency in traditional highway inspection.

[0017] 2. In this invention, a variational model integrating atmospheric scattering coefficient and motion blur kernel is constructed through the data processing module. Combined with the ADMM iterative algorithm, degraded images are specifically restored, eliminating interference factors such as scattering and blurring. The intelligent recognition module introduces Gabor filtering direction enhancement and spectral constraint mechanisms to inject physical prior knowledge into the deep convolutional neural network, enhancing its adaptability to complex environments, reducing the defect false detection rate and false judgment rate in complex scenarios, and improving the reliability of the detection results.

[0018] 3. In this invention, Gabor filtering is used to capture the directional features of cracks, a lightweight Transformer is used to characterize the global continuity of defects, and a multi-scale loss function is used to balance classification accuracy and physical rationality. Combined with the image output by the data processing module, the ability to identify subtle defects is improved. At the same time, the matching output module uses a projection transformation algorithm to accurately map the defect pixel coordinates to the latitude and longitude of the road map, ensuring the accuracy of defect location and providing accurate disease information for highway maintenance. Attached Figure Description

[0019] Figure 1 This is an architecture diagram of the highway detection device based on a UAV and PDA road map system proposed in this invention; Figure 2 This is a flowchart of the highway detection method based on UAV and PDA road map system proposed in this invention. Detailed Implementation

[0020] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Please see the appendix Figure 1 This invention provides a highway detection device based on a drone and a PDA road map system, comprising: Control Module: Used to import electronic maps into the PDA road map system, match the drone's flight trajectory with the PDA road map system, and plan the drone's flight path; Furthermore, the control provision module includes: Electronic map import unit: Used to import electronic maps into the PDA road map system, generate a highway network topology structure that includes road centerlines, curve curvature and road surface material distribution. The nodes of the highway network topology structure are road intersections, the edges are road segments, and it is accompanied by geometric coordinates and attribute labels. The trajectory matching unit is used to perform coordinate matching between the preset flight trajectory of a UAV and the centerline of a road network, based on the road network topology, to generate a set of geotagged path points. The coordinate matching satisfies X max =T(X) drone ), where X max For electronic map coordinates, X drone Here, T represents the UAV's GPS coordinates, and T(·) is the affine transformation function whose parameters are pre-calibrated by the PDA roadmap system. The parameter calculation unit is used to calculate the UAV's flight parameters based on the path point set and preset detection specifications, and to generate a flight path containing 3D path instructions. The detection specifications include the flight altitude range, flight speed range, camera exposure time, and image acquisition interval. It also calculates the length of the motion blur kernel in the UAV's flight parameters. Where v is the drone's flight speed, Δt is the camera exposure time, s is the camera pixel size, and h is the flight altitude.

[0022] Specifically, the control module is used to import electronic maps into the PDA roadmap system, match the UAV's flight trajectory with the PDA roadmap system, and plan the UAV's flight path. It includes an electronic map import unit, a trajectory matching unit, and a parameter calculation unit.

[0023] In some embodiments, the electronic map import unit is used to import the electronic map into the PDA road map system, generating a highway network topology that includes road centerlines, curve curvature, and pavement material distribution. Generally, the nodes of the highway network topology are road intersections, and the edges are road segments, accompanied by geometric coordinates and attribute labels. For example, the geometric coordinates include the latitude and longitude information of the start and end points of the road segment, and the attribute labels record information such as the pavement material and design speed of the road segment. Pavement materials include asphalt and cement.

[0024] As an alternative, the trajectory matching unit is used to perform coordinate matching between the pre-set flight trajectory of a UAV and the centerline of a road network, based on the road network topology, to generate a set of geotagged path points. In one possible implementation, the coordinate matching satisfies X max =T(X) drone ), where X max For electronic map coordinates, X drone Here, T(·) represents the UAV's GPS coordinates, and T(·) is an affine transformation function whose parameters are pre-calibrated by the PDA road map system. Specifically, the affine transformation function T(·) is calibrated by collecting the map coordinates of several known control points within the road map system and the UAV's GPS coordinates to ensure that the matching error between the flight trajectory and the road centerline is within a preset range.

[0025] In one possible implementation, the parameter calculation unit is used to calculate the UAV flight parameters based on the path point set and a preset detection specification, and generate a flight path containing 3D path instructions. Generally, the detection specification includes the flight altitude range, flight speed range, camera exposure time, and image acquisition interval. Specifically, when calculating the motion blur kernel length L in the UAV flight parameters, the UAV flight speed v, camera exposure time Δt, camera pixel size s, and flight altitude h are used as inputs, and the result is calculated using the formula... The motion blur kernel length L is calculated and used by the subsequent data processing module to perform motion blur restoration on the road surface images acquired by the UAV. For example, when the UAV's flight speed v is 10 m / s, the camera exposure time Δt is 1 / 500 s, the camera pixel size s is 2.4 × 10 m / pixel, and the flight altitude h is 8 m, the motion blur kernel length L can be calculated by substituting the values ​​into the formula. This value is used to guide the data processing module in building a motion fuzzy kernel model.

[0026] Drone data acquisition module: used to acquire road surface images based on the flight path of a drone equipped with multiple sensors; further, the drone data acquisition module includes: Multi-sensor control and acquisition unit: used to synchronously trigger multiple sensors according to the flight path to align data timestamps. The multiple sensors include an RGB camera, a multispectral sensor, and a GPS / IMU. Parameter synchronization unit: Used to collect road surface data including visible light images of the road surface, reflectivity data, real-time positioning data and attitude data using multiple sensors, and perform spatial registration to generate a road surface image with geographic coordinates.

[0027] Specifically, the drone acquisition module is used to acquire road surface images based on the flight path of a drone equipped with multiple sensors; it includes a multi-sensor control and acquisition unit and a parameter synchronization unit.

[0028] In some embodiments, the multi-sensor control and acquisition unit is used to synchronously trigger multiple sensors according to the flight path, aligning data timestamps. Typically, the multiple sensors include an RGB camera, a multispectral sensor, and a GPS / IMU. Alternatively, the waypoint sequence included in the flight path serves as a trigger reference. When the UAV flies to a preset spatial location of a waypoint, the multi-sensor control and acquisition unit generates a synchronization pulse through a hardware trigger circuit, driving the RGB camera, multispectral sensor, and GPS / IMU to start operating simultaneously. This ensures that the timestamp deviations of the visible light image of the road surface acquired by the RGB camera, the reflectivity data acquired by the multispectral sensor, and the real-time positioning and attitude data acquired by the GPS / IMU are within the microsecond range. For example, the RGB camera's imaging resolution is configured to 5472×3648 pixels to capture subtle road surface textures; the multispectral sensor has five detection channels covering the spectral range of 450nm to 900nm, used to extract the spectral reflectance characteristics of the road surface material.

[0029] In one possible implementation, a parameter synchronization unit is used to acquire road surface data using multiple sensors and perform spatial registration to generate a road surface image with geographic coordinates. Specifically, the road surface data includes grayscale images output by RGB cameras, band reflectivity matrices output by multispectral sensors, latitude, longitude, and altitude information output by GPS, and heading, pitch, and roll angles output by IMU. The parameter synchronization unit first calculates the image's shooting attitude using the IMU's attitude data, and then combines this with GPS positioning data and flight altitude to construct a projection transformation relationship from pixel coordinates to geographic coordinates. For example, it corrects the image's tilt deviation based on the IMU's roll and pitch angles, determines the image's orientation using the heading angle, calculates the geographic coordinates of the image center using GPS latitude, longitude, and altitude, and finally maps each image pixel to the geographic coordinate system, achieving spatial registration of the road surface image. This road surface image with geographic coordinates directly serves as the input source for the data processing module, providing position and attitude parameter support for subsequent variational model construction of motion blur kernels and estimation of atmospheric scattering coefficients.

[0030] Communication processing module: used for real-time data transmission and interactive operations; Furthermore, the communication processing module includes: Data Packaging Unit: Used to divide various data items into frames, add timestamps and check codes, and generate a data frame sequence; Dual-channel switching unit: used to monitor the signal strength of the 4G / 5G main channel in real time, and switch to the Bluetooth backup channel when the signal is below the threshold for data transmission.

[0031] Specifically, the communication processing module is used to transmit various data in real time and perform interactive operations, including a data packaging unit and a dual-channel switching unit.

[0032] In some embodiments, the data packaging unit is used to segment various data items into frames, add timestamps and checksums, and generate a data frame sequence. Generally, the data items encompass multi-source data such as road surface images with geographic coordinates output by the UAV acquisition module, flight parameters, and path instructions output by the control module. Alternatively, the frame segmentation strategy dynamically adapts to the data type and the MTU of the transmission channel. For example, RGB image data is spatially segmented into 1024×1024 pixel blocks, and structured data such as flight parameters is segmented into fixed-length fields. Specifically, the timestamp uses system absolute time encoding with millisecond-level precision to support the timing alignment logic of subsequent data processing modules; the checksum is generated using the CRC16 algorithm to perform redundancy checks on the data within the frame to detect data distortion in the transmission link. For example, when processing an RGB image with a resolution of 5472×3648, the data packaging unit first divides it into 18×12 1024×1024 pixel blocks, with the edge blocks adaptively adjusting their size. A 64-bit timestamp and a 16-bit CRC checksum are added to each pixel block, and it is encapsulated into a standard data frame to ensure data integrity and time-series traceability, thus meeting the data processing module's requirement for accurate parsing of the original data.

[0033] In one possible implementation, the dual-channel switching unit monitors the 4G / 5G primary channel signal strength in real time. When the signal falls below a threshold, it switches to the Bluetooth backup channel for data transmission. Generally, the primary channel prioritizes high-volume data such as high-definition images and batch parameters, while the backup Bluetooth channel is dedicated to transmitting low-latency data such as critical flight parameters and control commands. Specifically, the signal strength threshold is preset to -90dBm, dynamically monitored by real-time sampling of the primary channel's RSSI. When the RSSI remains below the threshold for more than 500 milliseconds, the channel switching process is triggered. Alternatively, the switching process follows a "connect first, disconnect later" logic: the dual-channel switching unit first establishes a connection to the Bluetooth backup channel, synchronously transmitting status indicators, and then interrupts data transmission on the primary channel, ensuring continuous data frame transmission without packet loss. For example, when a drone flies to a weak network area such as a tunnel entrance, the 4G / 5G main channel signal attenuates to -95dBm. The dual-channel switching unit automatically starts the Bluetooth connection and switches the transmission channel of key parameters such as flight speed and altitude to Bluetooth. This ensures that the interaction commands between the control module and the drone acquisition module can be reached in real time, and supports the stable operation of processes such as motion fuzzy kernel calculation and atmospheric scattering coefficient estimation of the data processing module.

[0034] Data processing module: used to restore road surface images based on variational models, eliminate the effects of image degradation in complex environments, and obtain processed data; Furthermore, the data processing module includes: Degradation parameter estimation unit: used to calculate the atmospheric scattering coefficient of road surface images based on multispectral data, and to calculate the motion blur kernel of road surface images based on GPS speed and camera parameters; Variational model building unit: used to build a variational model containing data fidelity terms, TV regularization terms, and nonlocal mean terms based on atmospheric scattering coefficients and motion fuzzy kernels; ADMM solver unit: used to iteratively solve variational models using the alternating direction multiplier method to generate non-degenerate processed data.

[0035] Furthermore, the atmospheric scattering coefficient is Where t is the mean transmittance and h is the flight altitude; the motion fuzzy kernel is... in, Let v be the length of the motion blur kernel, v be the real-time GPS speed of the UAV, Δt be the camera exposure time, s be the camera pixel size, and rect(·) be the rectangular function that satisfies... u represents the pixel displacement; The variational model is: Where J(x,y) is the restored image, A is the atmospheric light component, and k(u) is the motion blur kernel. rect(·) is a rectangle function. Let N(x,y) be the pixel block, λ and γ be the regularization coefficients, t be the mean transmittance, and w be the similarity weight. ′ ,y ′ ) represents the pixel coordinates paired with (x,y) in the nonlocal mean term; The alternating direction multiplier method is as follows: U k+1 =U k +J k+1 +Z k+1 ; The convergence condition is ||J k+1 -J k ||2<10 -4 , where J k Z is the restored image for the k-th iteration. k Auxiliary variable, U k Let ρ be the dual variable and ρ be the penalty parameter. The proximal operator for the TV regularization term.

[0036] Specifically, the data processing module is used to restore road surface images based on variational models, eliminate the effects of image degradation under complex environments, and obtain processed data; it includes a degradation parameter estimation unit, a variational model construction unit, and an ADMM solution unit.

[0037] In some embodiments, the degradation parameter estimation unit is used to calculate the atmospheric scattering coefficient of the road surface image based on multispectral data, and to calculate the motion blur kernel of the road surface image based on GPS speed and camera parameters. Generally, the multispectral data is provided by the multispectral sensor of the UAV acquisition module, containing reflectivity information for different bands; the GPS speed and camera parameters are output by the parameter synchronization unit of the UAV acquisition module and the parameter calculation unit of the control supply module, respectively. Specifically, when calculating the atmospheric scattering coefficient, the average transmittance t of the 700nm band of the multispectral image and the flight altitude h are used as inputs, and the formula is used... The atmospheric scattering coefficient β is obtained. For example, when the mean transmittance t is 0.6 and the flight altitude h is 8m, substituting into the formula yields... This value describes the degree to which the atmosphere scatters light, supporting the atmospheric light composition correction logic in the variational model building unit.

[0038] When calculating the motion blur kernel, the UAV's real-time GPS speed v, camera exposure time Δt, camera pixel size s, and flight altitude h are used as inputs. First, the kernel is calculated using the formula... Calculate the length L of the motion blur kernel, and then construct the motion blur kernel. Where rect(·) is a rectangle function, satisfying u is the pixel displacement. For example, when L = 6 × 10 -9 Then, the motion blur kernel k(u) is obtained, which is used to simulate the image blur caused by the motion of the UAV, and provides a basis for modeling the degradation process of the variational model.

[0039] Alternatively, the variational model building unit is used to construct a variational model containing data fidelity terms, TV regularization terms, and nonlocal mean terms based on the atmospheric scattering coefficient β and the motion blur kernel k(u). Generally, the data fidelity term is used to constrain the difference between the restored and degraded images, the TV regularization term is used to preserve image edges, and the nonlocal mean term is used to suppress noise. Specifically, the variational model is: Where I represents the degraded road surface image output by the UAV acquisition module, J represents the restored image to be solved, t represents the mean transmittance, calculated by the degradation parameter estimation unit, A represents the atmospheric light component, extracted from I using the dark channel priority algorithm, and k represents the motion blur kernel, calculated by the degradation parameter estimation unit. Let J be the gradient, and w be the similarity weight, expressed by the formula... The calculations show that N(x,y) is a 3×3 pixel block, σ is the standard deviation of the Gaussian kernel, and λ and γ are regularization coefficients, taken as 0.01 and 0.05 respectively. This model integrates multiple physical parameters to achieve joint modeling of complex degradations such as rain, fog, and motion blur, providing optimization objectives for the ADMM solver unit.

[0040] In one possible implementation, the ADMM solver unit is used to iteratively solve the variational model using the alternating direction multiplier method to generate degenerate processing data. Specifically, the iterative process includes: initializing J... 0 =I represents the degraded image. 0 is an auxiliary variable, U 0 =0 is the dual variable; then iteratively update. U k+1 =U k +J k+1 +Z k+1 until the convergence condition is met ||J k+1 -J k ||2<10 -4 Where ρ is the penalty parameter, set to 1.0. The proximal operator for the TV regularization term is implemented using fast gradient descent. For example, in the 20th iteration, if J... 20 With J 19 The L2 norm difference is less than 10 -4 If the iteration stops, output J. 20 As the restored processed data, this data can be directly input into the intelligent recognition module to support subsequent processes such as crack direction feature enhancement and false positive area filtering.

[0041] Intelligent recognition module: used to analyze the processed data and identify road surface defects by fusing physical priors into a deep convolutional neural network; Furthermore, the intelligent recognition module includes: Physical feature extraction unit: used to enhance crack direction features in the processed data through Gabor filtering, and to filter out false positive regions in the processed data through spectral constraints, thereby extracting physical feature data that incorporates physical priors; Semantic Analysis Unit: Used to capture the continuity of cracks in the processed data through a lightweight Transformer to generate a semantic feature map of defects; Fusion Decision Unit: Used to fuse physical feature data and defect semantic feature map through attention mechanism, and optimize based on multi-scale loss function in deep convolutional neural network to output road surface defects, which include type, location, features and confidence level.

[0042] Furthermore, the formula for the Gabor filter kernel is: Where (x,y) are the pixel coordinates of the processed data, θ is the principal direction of the crack, f is the spatial frequency, and σ is the pixel coordinates of the processed data. x σ y Let be the Gaussian kernel standard deviation, exp(i2πfx) ′ ) represents a complex exponential term; The attention formula for lightweight Transformer is: Where Q, K, and V represent the query, key, and value projection of the physical feature data, respectively. T To calculate the similarity between features, The scaling factor, softmax(·) is the normalization function; The attention mechanism is F fusion =Attention(F phys F sem F sem ), where F phys For physical characteristic data, F sem For the defect semantic feature map, F fusion To integrate physical feature data with features associated with the defect semantic feature map; The multi-scale loss function is Where α, β, and γ are the loss weights. For Focal Loss, For physical constraint loss, This is a bi-branch feature consistency loss.

[0043] Specifically, the intelligent recognition module is used to analyze the processed data and identify road surface defects by fusing physical priors into a deep convolutional neural network; it includes a physical feature extraction unit, a semantic analysis unit, and a fusion decision unit.

[0044] In some embodiments, the physical feature extraction unit is used to enhance crack orientation features in the processed data through Gabor filtering and to filter false positive regions in the processed data through spectral constraints, thereby extracting physical feature data that incorporates physical priors. Generally, the processed data is output by the ADMM solving unit of the data processing module and includes a restored, clear road surface image. Specifically, the Gabor filter kernel formula is as follows: Where (x,y) are the pixel coordinates of the processed data, θ∈{0°,45°,90°,135°} is the main direction of the crack, adapted to the common orientation of highway cracks, f=0.2 is the spatial frequency, and σ x =2.0, σ y =1.0 is the Gaussian kernel standard deviation, x ′ =xcosθ+ysinθ, y ′=-xsinθ+ycosθ represents a coordinate rotation transformation. For example, when θ=45°, the filter kernel enhances the texture response of the diagonal crack, making the crack edges more prominent. The spectral constraint formula is... Where R(p) is the 700nm band reflectance collected by the multispectral sensor, which comes from the UAV acquisition module, and R0 = 0.35 is the average healthy pavement reflectance pre-stored by the control module. For example, when the pavement area has R(p) significantly lower than R0 due to shadows, C(p) = 1 is marked as a defect candidate area, filtering out false positive interference from vegetation, highlights, etc., and finally outputting physical feature data that fuses direction and spectral characteristics.

[0045] Alternatively, the semantic analysis unit is used to capture the continuity of cracks in the processed data using a lightweight Transformer, generating a semantic feature map of defects. Typically, the processed data, after initial processing by the physical feature extraction unit, is input into the deep convolutional layer of the semantic analysis unit. Specifically, the attention formula of the lightweight Transformer is... Where Q, K, and V are the query, key, and value matrices respectively after 1×1 convolution projection of the physical feature data, and d k =64 is the feature dimension, QK T Calculate the similarity between features. The scaling factor is softmax(·), which normalizes the attention weights. For example, crack pixels in physical feature data are associated with distant pixels through an attention mechanism, enhancing the expression of crack continuity and generating a defect semantic feature map with dimensions matching the input, thus compensating for the locality limitations of Gabor filtering.

[0046] In one possible implementation, the fusion decision unit is used to fuse physical feature data and defect semantic feature maps through an attention mechanism, and optimizes the result based on a multi-scale loss function in a deep convolutional neural network to output the road surface defect. Specifically, the attention mechanism formula is F. fusion =Attention(F phys F sem F sem ), where F phys F represents the physical feature data output by the physical feature extraction unit. sem The semantic feature map of the defect output by the semantic analysis unit is compressed to the same channel by a 1×1 convolution. Weights are then dynamically assigned through attention to fuse local details and global structural features. The multi-scale loss function is... Where α = 1.0, β = 0.5, and γ = 0.2 are the loss weights. For Focal Loss, the input is a classification prediction based on fused features. The physical constraint loss is used to correlate the motion blur kernel of the data processing module, penalizing misjudgments in blurry regions. The dual-branch feature consistency loss constrains the similarity between physical and semantic features. For example, during iterative optimization, the multi-scale loss function guides the network to simultaneously learn the classification boundary, physical rationality, and feature consistency of defects, ultimately outputting road surface defects containing type, location, features, and confidence level, supporting coordinate association and report generation in the matching output module.

[0047] Matching output module: Used to match road surface defects with coordinates in the PDA road map system to generate an inspection report containing defect information.

[0048] Furthermore, the matching output module includes: Coordinate transformation unit: used to map the image pixel coordinates of road surface defects to the latitude and longitude of the electronic map through projection transformation algorithm, and generate defect locations with geographic labels; Report generation unit: Used to classify defect locations according to defect characteristics and generate structured reports that include location, type, size, and severity; Visualization output unit: used to render a defect heatmap in the PDA roadmap system based on the structured report, display the defect level through color coding, and generate an inspection report containing defect information.

[0049] Specifically, the matching output module is used to match road surface defects with coordinates in the PDA road map system to generate an inspection report containing defect information, including a coordinate transformation unit, a report generation unit, and a visualization output unit.

[0050] In some embodiments, the coordinate transformation unit is used to map the image pixel coordinates of road surface defects to the latitude and longitude of an electronic map using a projection transformation algorithm, generating a geographically tagged defect location. Generally, the image pixel coordinates are output by the intelligent recognition module and include the pixel coordinates of the defect edges and center; the parameters of the projection transformation are pre-calibrated by the trajectory matching unit of the control module. Specifically, the projection transformation satisfies the formula... Where (x) p y p Let (x, y) be the image pixel coordinates of the road surface defect, H be a 3×3 projection matrix obtained by calibrating the map coordinates of several control points within the road map system with the UAV image coordinates, and (Lon, Lat) be the latitude and longitude of the mapped electronic map. For example, the intelligent recognition module outputs the edge pixel coordinates (x, y) of a crack. p =100, y p Substitute the pre-stored projection matrix H into the matrix (Lon = 200) to calculate the corresponding latitude and longitude (Lon = 116.4075, Lat = 39.9042), generate the defect location with geographic tags, and provide spatial positioning basis for the report generation unit.

[0051] Alternatively, the report generation unit is used to classify defect locations based on defect characteristics, generating a structured report containing location, type, size, and severity. Generally, the defect characteristics are output by the intelligent recognition module, covering crack length, pit diameter, and defect type, such as cracks or pits. Specifically, crack length is calculated using the formula L = ||p1-p2||2·s·h / f, where p1 and p2 are the pixel coordinates of the crack edge, s is the camera pixel size (from the parameter calculation unit of the control module), h is the flight altitude (from the control module), and f is the camera focal length (a hardware calibration parameter). Pit diameter is calculated similarly, by measuring the distance between defect edge pixels and converting it to the actual size. Classification is based on preset thresholds; for example, a crack length > 50cm is classified as a level 3 defect. Defects are divided into levels 1-5, ultimately generating a structured report containing latitude and longitude location, defect type, size, and severity, providing data support for the visualization output unit.

[0052] In one possible implementation, the visualization output unit renders a defect heatmap in the PDA roadmap system based on the structured report, displays the defect level using color coding, and generates an inspection report containing defect information. Generally, the severity of the structured report corresponds to the color coding; for example, red represents a level 5 defect, yellow represents a level 3 defect, and green represents a level 1 defect. Specifically, the heatmap is generated using a formula... Rendering, where (x i ,y i ) represents the geographic coordinates of the defect, derived from the coordinate transformation unit, σ M =0.5m is the Gaussian kernel standard deviation, which makes the defect area present a continuous thermal distribution. For example, the geographical coordinates of a level 5 pothole (Lon=116.4075, Lat=39.9042) correspond to the red highlighted area on the heat map. Combined with the text information in the structured report, a complete inspection report is generated, realizing the visualization of road defects and data archiving.

[0053] The control module imports electronic maps into the PDA road map system and automatically plans drone flight paths, eliminating the need for manual on-site surveys and path design. The drone acquisition module, equipped with multiple sensors, synchronously triggers data acquisition based on the planned path, achieving one-time acquisition of road surface images and related parameters, avoiding the time loss of step-by-step acquisition. The communication processing module transmits data in real-time via a dual-channel mechanism, ensuring seamless integration of acquisition and processing, and reducing data transmission latency. The data processing module uses a variational model to restore road surface images, automatically eliminating image degradation caused by complex environments. The intelligent recognition module analyzes the processed data using a deep convolutional neural network that integrates physical priors, automatically identifying road surface defects and replacing manual interpretation. The matching output module matches road surface defects with coordinates in the PDA road map system and automatically generates an inspection report. Through the automated collaborative work of these modules, a fully automated inspection system is constructed, encompassing path planning, data acquisition, transmission, processing, recognition, and report generation. This achieves efficient highway inspection operations and solves the problem of low efficiency in traditional highway inspection. Please see the appendix Figure 2 A highway detection method based on UAV and PDA road map system, applied to the aforementioned highway detection device based on UAV and PDA road map system, includes the following steps: Control provision: Import electronic maps into the PDA road map system, match the drone's flight trajectory with the PDA road map system, and plan the drone's flight path; Drone data collection: Using drones equipped with multiple sensors to collect road surface images based on their flight paths; Communication processing: Real-time transmission of various data and interactive operation; Data processing: The road surface image is restored based on the variational model to eliminate the image degradation effect under complex environment and obtain the processed data; Intelligent recognition: By analyzing the processed data through a deep convolutional neural network that integrates physical priors, road surface defects are identified; Matching output: The road surface defects are matched with the coordinates in the PDA road map system to generate a detection report containing defect information.

[0054] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A highway inspection device based on a UAV and PDA road map system, characterized in that, include: Control Module: Used to import electronic maps into the PDA road map system, match the drone's flight trajectory with the PDA road map system, and plan the drone's flight path; Drone data acquisition module: Used to acquire road surface images based on the flight path of a drone equipped with multiple sensors; Communication processing module: used for real-time data transmission and interactive operations; Data processing module: used to restore road surface images based on variational models, eliminate the effects of image degradation in complex environments, and obtain processed data; Intelligent recognition module: used to analyze the processed data and identify road surface defects by fusing physical priors into a deep convolutional neural network; Matching output module: Used to match road surface defects with coordinates in the PDA road map system to generate an inspection report containing defect information.

2. The highway inspection device based on a UAV and PDA road map system according to claim 1, characterized in that, The control providing module includes: Electronic map import unit: used to import electronic maps into the PDA road map system, generate a highway network topology structure including road centerlines, curve curvature and road surface material distribution. The nodes of the highway network topology structure are road intersections, the edges are road segments, and it is accompanied by geometric coordinates and attribute labels. Trajectory matching unit: Used to perform coordinate matching between the preset flight trajectory of the UAV and the centerline of the road based on the road network topology, generating a set of geotagged path points, wherein the coordinate matching satisfies X max =T(X) drone ), where X max For electronic map coordinates, X drone Here are the UAV's GPS coordinates, and T(·) is the affine transformation function whose parameters are pre-calibrated by the PDA roadmap system; Parameter calculation unit: Used to calculate UAV flight parameters based on a path point set and preset detection specifications, and generate a flight path containing 3D path instructions. The detection specifications include flight altitude range, flight speed range, camera exposure time, and image acquisition interval. The length of the motion blur kernel in the calculation of UAV flight parameters... Where v is the drone's flight speed, Δt is the camera exposure time, s is the camera pixel size, and h is the flight altitude.

3. The highway inspection device based on a UAV and PDA road map system according to claim 1, characterized in that, The UAV data acquisition module includes: Multi-sensor control and acquisition unit: used to synchronously trigger multiple sensors according to the flight path to align data timestamps. The multiple sensors include an RGB camera, a multispectral sensor, and a GPS / IMU. Parameter synchronization unit: Used to collect road surface data including visible light images of the road surface, reflectivity data, real-time positioning data and attitude data using multiple sensors, and perform spatial registration to generate a road surface image with geographic coordinates.

4. The highway inspection device based on a UAV and PDA road map system according to claim 1, characterized in that, The communication processing module includes: Data Packaging Unit: Used to divide various data items into frames, add timestamps and check codes, and generate a data frame sequence; Dual-channel switching unit: used to monitor the signal strength of the 4G / 5G main channel in real time, and switch to the Bluetooth backup channel when the signal is below the threshold for data transmission.

5. The highway inspection device based on a UAV and PDA road map system according to claim 1, characterized in that, The data processing module includes: Degradation parameter estimation unit: used to calculate the atmospheric scattering coefficient of road surface images based on multispectral data, and to calculate the motion blur kernel of road surface images based on GPS speed and camera parameters; Variational model building unit: used to build a variational model containing data fidelity terms, TV regularization terms, and nonlocal mean terms based on atmospheric scattering coefficients and motion fuzzy kernels; ADMM solver unit: used to iteratively solve variational models using the alternating direction multiplier method to generate non-degenerate processed data.

6. The highway inspection device based on a UAV and PDA road map system according to claim 5, characterized in that, The atmospheric scattering coefficient is Where t is the average transmittance and h is the flight altitude; The motion blur kernel is in, Let v be the length of the motion blur kernel, v be the real-time GPS speed of the UAV, Δt be the camera exposure time, s be the camera pixel size, and rect(·) be the rectangular function that satisfies... u represents the pixel displacement; The variational model is: Where J(x,y) is the restored image, A is the atmospheric light component, and k(u) is the motion blur kernel. rect(·) is a rectangle function. Let N(x,y) be the pixel block, λ and γ be the regularization coefficients, t be the mean transmittance, and w be the similarity weight. ′ ,y ′ ) represents the pixel coordinates paired with (x,y) in the nonlocal mean term; The alternating direction multiplier method is as follows: U k+1 =U k +J k+1 +Z k+1 ; The convergence condition is ||J k+1 -J k ||2<10 -4 , where J k Z is the restored image for the k-th iteration. k Auxiliary variable, U k Let ρ be the dual variable and ρ be the penalty parameter. The proximal operator for the TV regularization term.

7. The highway inspection device based on a UAV and PDA road map system according to claim 1, characterized in that, The intelligent recognition module includes: Physical feature extraction unit: used to enhance crack direction features in the processed data through Gabor filtering, and to filter out false positive regions in the processed data through spectral constraints, thereby extracting physical feature data that incorporates physical priors; Semantic Analysis Unit: Used to capture the continuity of cracks in the processed data through a lightweight Transformer to generate a semantic feature map of defects; Fusion Decision Unit: Used to fuse physical feature data and defect semantic feature map through attention mechanism, and optimize based on multi-scale loss function in deep convolutional neural network to output road surface defect, which includes type, location, features and confidence level.

8. The highway inspection device based on a UAV and PDA road map system according to claim 7, characterized in that, The Gabor filter kernel formula is as follows: Where (x,y) are the pixel coordinates of the processed data, θ is the principal direction of the crack, f is the spatial frequency, and σ is the spatial frequency. x σ y Let be the Gaussian kernel standard deviation, exp(i2πfx) ′ ) represents a complex exponential term; The attention formula for the lightweight Transformer is as follows: Where Q, K, and V represent the query, key, and value projection of the physical feature data, respectively. T To calculate the similarity between features, The scaling factor, softmax(·) is the normalization function; The attention mechanism is F fusion =Attention(F phys F sem F sem ), where F phys For physical characteristic data, F sem For the defect semantic feature map, F fusion To integrate physical feature data with features associated with defect semantic feature maps; The multi-scale loss function is: Where α, β, and γ are the loss weights. For Focal Loss, For physical constraint loss, This is a bi-branch feature consistency loss.

9. The highway inspection device based on a UAV and PDA road map system according to claim 1, characterized in that, The matching output module includes: Coordinate transformation unit: used to map the image pixel coordinates of road surface defects to the latitude and longitude of the electronic map through projection transformation algorithm, and generate defect locations with geographic labels; Report generation unit: Used to classify defect locations according to defect characteristics and generate structured reports that include location, type, size, and severity; Visualization output unit: used to render a defect heatmap in the PDA roadmap system based on the structured report, display the defect level through color coding, and generate an inspection report containing defect information.

10. A highway detection method based on UAV and PDA road map system, characterized in that, The highway inspection device based on the UAV and PDA road map system as described in any one of claims 1-9 includes the following steps: Control provision: Import electronic maps into the PDA road map system, match the drone's flight trajectory with the PDA road map system, and plan the drone's flight path; Drone data collection: Using drones equipped with multiple sensors to collect road surface images based on their flight paths; Communication processing: Real-time transmission of various data and interactive operation; Data processing: The road surface image is restored based on the variational model to eliminate the image degradation effect under complex environment and obtain the processed data; Intelligent recognition: By analyzing the processed data through a deep convolutional neural network that incorporates physical priors, road surface defects can be identified; Matching output: Matches road surface defects with coordinates in the PDA road map system to generate an inspection report containing defect information.

Images