A multi-source positioning information fusion full-section unmanned aerial vehicle intelligent highway inspection method

Abstract

This invention discloses a method for intelligent highway inspection using unmanned aerial vehicles (UAVs) based on multi-source positioning information fusion. The steps are as follows: Based on IMU inertial recursion, the optimal state estimation results of the UAV based on GNSS data and UWB data are obtained respectively; the globally optimal state estimation model is used to obtain the globally optimal state estimation of the UAV; a road defect identification network model is constructed to process the continuous video frames of the UAV inspection frame by frame, and outputs the image pixel coordinates containing the road defect category and the image of the road defect detection box; an inverse perspective mapping model is constructed to obtain the three-dimensional physical coordinates of the road defects; redundant results are removed using sliding window majority voting logic, and combined with spatial dimension merging and clustering, to output the effective three-dimensional physical coordinates of the road defects. This method overcomes the limitations of two-dimensional detection, has high reliability, no blind spots in positioning, can effectively resist false positive detections caused by interference such as lighting, filters duplicate markings generated by the same road defect, and has high accuracy.

Description

Technical Field

[0001] This invention relates to the field of highway health monitoring and unmanned aerial vehicle (UAV) inspection technology, and in particular to a method for intelligent UAV inspection of the entire road segment using multi-source positioning information fusion. Background Technology

[0002] As highways age, surface defects such as cracks and potholes inevitably appear. Early detection and preventative maintenance of these defects are crucial for extending highway lifespan and reducing large-scale repair costs.

[0003] In recent years, drone inspection methods have been gradually introduced into highway maintenance systems. For example, the published patent CN121214261A provides a multi-target recognition method suitable for daily drone highway maintenance inspections, including determining the inspection target of the drone highway inspection, arranging the drone's image acquisition flight plan, and acquiring image data of asphalt pavement defects or traffic infrastructure damage. However, this method directly relies on image overlap for coordinate calculation, does not rely on high-precision positioning equipment, has a large error, and does not support real-time inspection. The published patent CN121577031A provides a highway inspection target positioning method based on multi-source data fusion. This method obtains fused navigation data through deep coupling of satellite positioning and inertial navigation and Kalman filtering correction; then it fuses two-dimensional vision and three-dimensional point cloud data to generate a front view image depth map, and calculates the national geodetic coordinates of the defects; finally, it combines the starting kilometer marker and uses Gaussian forward calculation to determine the mileage marker of the defects. However, this method does not use indoor positioning devices such as UWB and requires coordinate transformation based on three-dimensional point cloud maps, making it impossible to achieve low-cost, high-precision defect positioning in satellite-denied environments.

[0004] Based on this, in order to achieve accurate location and real-time early warning of road defects across the entire road section, it is necessary to design a method that integrates multi-source positioning and sensing data and optimizes the positioning solution method to meet the needs of low-cost, high-precision, and real-time inspection and positioning of road defects in satellite-denied environments. Summary of the Invention

[0005] The purpose of this invention is to provide a method for intelligent highway inspection using unmanned aerial vehicles (UAVs) that integrates multi-source positioning information to solve the above-mentioned technical problems.

[0006] Therefore, the technical solution of the present invention is as follows:

[0007] A method for intelligent highway inspection using unmanned aerial vehicles (UAVs) based on multi-source positioning information fusion, comprising the following steps:

[0008] S1. Utilize UAV-borne multi-source sensors to synchronously acquire RTK-GNSS data, UWB data, and IMU data. First, construct an inertial recursive prediction model and a GNSS / UWB measurement update model. Based on IMU inertial recursion, obtain the optimal state estimation results based on GNSS data and the optimal state estimation results based on UWB data, respectively, under the same spatial reference frame. Construct a global optimal state estimation model and use the optimal state estimation results based on GNSS data and the optimal state estimation results based on UWB data at time k under the unified ENU coordinate system to obtain the global optimal state estimation of the UAV at time k in the ENU coordinate system.

[0009] S2. Construct a road defect identification network model to process the continuous video frames collected during the UAV inspection process frame by frame, identify the road defects in each frame, and output the image pixel coordinates and road defect detection boxes containing the road defect categories.

[0010] S3. Construct an inverse perspective mapping model to combine the fusion position result of the UAV time k obtained in step S1, the camera offset, and the camera intrinsic parameters to convert the image pixel coordinates of the road defects obtained in step S2 into the three-dimensional physical coordinates of the road defects in the real world coordinate system.

[0011] S4. Using the sliding window majority voting logic, the three-dimensional physical coordinates of road defects mapped in the continuous frame video stream are effectively confirmed, and road defects belonging to the same source are spatially merged and clustered to output the effective three-dimensional physical coordinates of road defects.

[0012] Furthermore, in step S1, the method for obtaining the optimal state estimation results based on GNSS data and the optimal state estimation results based on UWB data at time k of the UAV is as follows:

[0013] 1) Construct an inertial recursive prediction model to obtain the inertial recursive state vector of the UAV at time k. The expression for the inertial recursive prediction model is:

[0014] ,

[0015] ,

[0016] In the formula, For the prior velocity estimate of the UAV at time k, Let be the posterior velocity of the drone at time k-1. Let k be the prior estimated position of the UAV at time k. Let k be the posterior position of the drone at time k-1;

[0017] in, The acceleration of the UAV in the navigation coordinate system is expressed as follows: , Let be the attitude matrix of the UAV at time k, from the carrier coordinate system to the ENU coordinate system, and its expression is: , , The three-axis angular velocities output by the IMU at time k. This refers to the data collection interval. For antisymmetric matrix operators; Let be the gravity vector in the ENU coordinate system. ; The three-axis specific force output by the IMU at time k;

[0018] Furthermore, the inertial recursive state vector of the UAV at time k is obtained. : ;

[0019] 2) Construct a GNSS / UWB measurement update model based on the UAV's inertial recursive state vector at time k. The optimal state estimates based on GNSS data and the optimal state estimates based on UWB data were obtained respectively.

[0020] The expression for the GNSS / UWB measurement update model is:

[0021] ,

[0022] ,

[0023] ,

[0024] ,

[0025] In the formula, Let be the prediction error covariance matrix at time k; Let be the updated error covariance matrix at time k-1; This is the state transition matrix; The process noise covariance matrix; Kalman gain; The noise covariance matrix is ​​set based on the prior accuracy of the sensor. Estimate the optimal state vector of the UAV at time k; Let be the inertial recursive state vector of the UAV at time k; Let be the observation matrix of the UAV at time k; Let be the observation vector of the UAV at time k; Let $k$ be the error covariance matrix of the UAV updated at time $k$.

[0026] 3) Construct the GNSS observation vector of the UAV at time k and GNSS observation matrix The data is then substituted into the GNSS / UWB measurement update model to obtain the optimal state estimate of the UAV based on GNSS data at time k. Construct the UWB observation vector of the UAV at time k. and UWB observation matrix The data is then substituted into the GNSS / UWB measurement update model to obtain the optimal state estimate of the UAV based on UWB data at time k. .

[0027] Furthermore, the GNSS observation vector of the UAV at time k and GNSS observation matrix The GNSS position and GNSS velocity of the UAV at time k are constructed, and their expressions are as follows:

[0028] ,

[0029] ,

[0030] In the formula, Let k be the coordinates of the UAV's GNSS position on the x-axis, y-axis, and z-axis in the WGS-84 coordinate system at time k. These represent the velocities in the x, y, and z directions of the UAV at time k, calculated based on RTK-GNSS data.

[0031] UWB observation vector of the UAV at time k and UWB observation matrix The UWB location is constructed based on the UAV time k, and the expressions for both are:

[0032] ,

[0033] ,

[0034] In the formula, Let k be the coordinates of the UWB position of the UAV at time k on the x-axis, y-axis, and z-axis of the ENU coordinate system.

[0035] Furthermore, in step S1, the optimal state estimation results based on GNSS data acquired by the UAV at consecutive time points are processed using a sliding window mid-value filter. And the optimal state estimation results based on UWB data. Smoothing is performed; the smoothing method for any optimal state estimation result of the UAV at time k is as follows: set the sliding window length to N, where N is an odd number; take time k as the center, obtain N consecutive optimal state estimation results from the past (N-1) / 2 sampling times to the future (N-1) / 2 sampling times; sort the N consecutive optimal state estimation results, and take the median value as the effective observation value of the current time k, so as to update the optimal state estimation result based on GNSS data and the optimal state estimation result based on UWB data at time k of the UAV.

[0036] Furthermore, in step S1, the method for obtaining the global optimal state estimate of the UAV at time k in the ENU coordinate system is as follows:

[0037] 1) Transform the optimal state estimation results based on GNSS data at time k of the UAV from the WGS-84 coordinate system to the ENU coordinate system;

[0038] 2) Based on the optimal state estimation results of the UAV at time k in the unified ENU coordinate system using GNSS data and UWB data, a global observation vector for UAV at time k is constructed. Correspondingly, the global observation matrix of the UAV at time k Designed as follows:

[0039] ;

[0040] 3) Construct a globally optimal state estimation model, the expression of which is:

[0041] ,

[0042] ,

[0043] ,

[0044] ,

[0045] ,

[0046] In the formula, Let be the global prediction error covariance matrix at time k; The global error covariance matrix is ​​updated at time k-1; This is the state transition matrix; The process noise covariance matrix; For global Kalman gain; The measurement noise covariance matrix; This is the global optimal state vector estimate for the UAV at time k; Let be the global inertial recursive state vector of the UAV at time k; Let be the global observation matrix of the UAV at time k; Let be the global observation vector of the UAV at time k; Let $k$ be the global error covariance matrix of the UAV updated at time $k$.

[0047] 4) Solve to obtain the global optimal state vector estimate of the UAV at time k. The global error covariance matrix of the UAV at time k. .

[0048] Furthermore, in step S2, the road defect identification network model is constructed based on an improved YOLOv8 network. The specific construction steps are as follows:

[0049] 1) Construct a flexible attention module, which uses global average pooling to obtain channel global feature descriptors, and then generates an adaptive weight matrix through two fully connected networks and activation functions. Finally, the adaptive weight matrix is ​​processed by performing a Hadamard product with the input feature map to output the attention-enhanced feature map.

[0050] 2) Construct a spatial downsampling module, which takes the input feature map and passes it through a step size of 2. Convolution performs spatial downsampling and utilizes channel attention for recombination to output an efficient downsampled feature map;

[0051] 3) Improvements to the YOLOv8 network include: replacing the convolutional modules in layers 0, 1, 3, 5, and 7 of the backbone network with spatial downsampling modules, and adding flexible attention modules at the outputs of the newly replaced spatial downsampling modules in layers 3, 5, and 7; replacing the convolutional modules in layers 16 and 19 of the head network with spatial downsampling modules, and adding flexible attention modules at the outputs of the newly replaced spatial downsampling modules in layers 16 and 19.

[0052] Furthermore, in step S3, the specific method for converting any road defect detection box in any image result output by the road defect recognition network model in step S2 is as follows:

[0053] S301. Extract the coordinates of the center point of the road defect detection frame. And this serves as the initial position of the road defect in the image pixel coordinate system;

[0054] S302. Obtain the fused pose of the UAV at time k in the ENU coordinate system, which is determined by its position. and attitude quaternions Composition, attitude quaternions Convert to standard Direction cosine matrix The translation offset vector of the camera's optical center relative to the UAV's body coordinate system is determined as follows: And the camera Euler angle bias, and convert the camera Euler angle bias into a relative rotation matrix. Then, using the rigid body kinematics equations, the absolute position of the camera's optical center in the ENU coordinate system can be obtained. With absolute attitude rotation matrix The expression for the relationship between them is:

[0055] ,

[0056] ;

[0057] Wherein, the fused pose of the UAV at time k in the ENU coordinate system is the estimated global optimal state vector of the UAV at time k obtained in step S1. The position components in the image are inversely transformed to their positions in the WGS-84 coordinate system.

[0058] S303, Based on the width of the image captured by the camera and horizontal field of view Calculate the camera's physical equivalent focal length. : ; with the center point of the image Using the origin as the initial point, the road defects are located in the image pixel coordinate system at their initial position (pixels). Represented as the ray direction vector in the camera coordinate system : ; for the ray direction vector After normalizing the magnitude, the unit direction vector is obtained. : ;

[0059] S304. The camera ray unit direction vector obtained from step S3023 is... Unit direction vector transformed to ENU coordinate system : In a highway inspection scenario, a local area of ​​the road surface is approximated as a horizontal plane, and the absolute elevation of this road surface in the ENU coordinate system is obtained. Using the ray parameter equation: Solving for the optical center of the camera yields the solution. Starting point A spatial ray with direction and a plane spatial intersection , The scaling parameter is obtained by expanding... The axis components are obtained by solving. , and The absolute elevations are respectively and the unit direction vector of the camera ray The component on the Z-axis; if and only if the scaling parameter When, find the spatial intersection point. That is, to obtain the three-dimensional physical coordinates of road defects in the real-world coordinate system. .

[0060] Furthermore, in step S302, the fused pose of the UAV at time k in the ENU coordinate system is the globally optimal state vector estimate of the UAV at time k obtained in step S1. The position components in the coordinate system are inversely transformed to longitude, latitude, and elevation in the WGS-84 coordinate system.

[0061] Furthermore, the specific implementation steps of step S4 are as follows:

[0062] S401. Define the sliding window majority voting logic, including: setting the time window length. The images processed in step S3 are randomly sampled sequentially within each time window; the maximum number of sampling frames within a single window is set. To determine the number of image frames randomly selected within each time window length. Set the minimum confidence threshold. This serves as the basis for judging the existence of road defects. The number of frames in the extracted images within the time window that detect the same type of road defect is considered. If so, the road defect is determined to be a validly detected road defect. ;

[0063] S402. Using spatial distance clustering, the set of valid disease points confirmed in step S401 is processed. The physical entities of road defects are spatially deduplicated, among which... This represents the i-th road defect point, which is determined by the three-dimensional physical coordinates of the road defect. and road disease attributes constitute, , This represents the attribute of the k-th type of road defect; This represents the total number of road defects;

[0064] S403. Multiple road disease points identified as belonging to the same source as determined in step S402 are merged into one, and the moving average method is used to update and calculate the three-dimensional physical coordinates of the multiple road disease points to output the three-dimensional physical coordinates of the effective road disease and its corresponding category attributes.

[0065] Furthermore, the specific operation steps of step S402 are as follows:

[0066] S4021, Collection of effective disease points All road defects are classified according to their attributes. Grouping;

[0067] S4022. For all road defect points in the same group, calculate the Euclidean distance between every two road defect points sequentially, and then apply the calculated distance based on the preset spatial clustering distance threshold. Determine whether they belong to the same source of road defects;

[0068] For any road defect point in the same group and The Euclidean distance between the two The calculation expression is:

[0069] ,

[0070] In the formula, and Road defects Three-dimensional physical coordinates The X and Y coordinates are in the middle. and Road defects Three-dimensional physical coordinates The X and Y coordinates are in the middle.

[0071] Furthermore, if If the two road defects are found to be of the same origin, then they are determined to be road defects from the same source; otherwise, they are determined to be road defects from different origins.

[0072] Compared with existing technologies, the beneficial effects of this multi-source positioning information fusion-based unmanned aerial vehicle (UAV) intelligent highway inspection method for the entire road segment are as follows:

[0073] (1) In step S1, the method of the present invention utilizes the mechanism of dynamically adjusting the measurement variance to adaptively adapt to GNSS and UWB signals, thereby realizing the smooth positioning of the UAV with zero jump and seamless switching between open road sections and satellite positioning blind zones. It has high reliability and no blind zone positioning. According to actual application tests, the method achieves outdoor static and dynamic accuracy of 0.01m.

[0074] (2) The method of the present invention breaks through the limitations of two-dimensional detection by using the image recognition technology in step S2 and the inverse perspective mapping in step S3. Combined with the inverse perspective transformation of the pose of a high-precision airborne camera, the high-precision physical coordinates of the disease can be obtained.

[0075] (3) The method of the present invention adds step S4 to design sliding window majority voting logic and adopts a dual filtering mechanism of "time dimension voting + spatial dimension clustering" to achieve extremely low data redundancy and false alarm rate. On the basis of meeting the real-time requirements, it can effectively resist false positive detection caused by interference such as light, filter duplicate marks generated by the same disease during dynamic flight, and achieve high disease deduplication rate and high positioning accuracy. Attached Figure Description

[0076] Figure 1 The flowchart is a process for the intelligent highway inspection method using unmanned aerial vehicles (UAVs) for the entire road segment based on multi-source positioning information fusion, as described in this invention.

[0077] Figure 2 This is a performance comparison chart of different filtering methods before and after optimization using the method of this application in an embodiment of the present invention, showing the rejection rate and false positive rate.

[0078] Figure 3 This is a performance comparison chart of different filtering methods before and after optimization using the method of this application in an embodiment of the present invention, showing the output of the number of defects. Detailed Implementation

[0079] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but the following embodiments are by no means intended to limit the present invention.

[0080] See Figure 1 The specific implementation steps of this multi-source positioning information fusion-based unmanned aerial vehicle (UAV) intelligent highway inspection method for the entire road segment are described below.

[0081] S1. Utilize UAV-borne multi-source sensors to synchronously acquire RTK-GNSS data, UWB data, and IMU data. First, construct an inertial recursive prediction model and a GNSS / UWB measurement update model. Based on IMU inertial recursion, obtain the optimal state estimation results based on GNSS data and the optimal state estimation results based on UWB data, respectively, under the same spatial reference frame. Construct a global optimal state estimation model and use the optimal state estimation results based on GNSS data and the optimal state estimation results based on UWB data at time k under the unified ENU coordinate system to obtain the global optimal state estimation of the UAV at time k in the ENU coordinate system.

[0082] Specifically, the implementation steps of step S1 are described as follows.

[0083] S101. Equip a multi-source sensor on a drone used for highway inspection to collect RTK-GNSS data, UWB data and IMU data in real time.

[0084] RTK-GNSS data specifically refers to UAV GNSS observation data obtained by receiving satellite signals in real time through a GNSS receiver and analyzing the satellite signals. Specifically, it includes the UAV's GNSS position, GNSS velocity, and timestamp. The UAV's GNSS position is specifically composed of longitude, latitude, and altitude.

[0085] UWB data specifically refers to raw UWB ranging data collected by at least three UWB base stations with known coordinates based on UWB tags deployed on UWB drones, and the UWB location and timestamp of the drone are calculated using the Time Difference of Arrival (TDOA) positioning system of the UWB module.

[0086] IMU data specifically refers to the raw inertial data of the drone during its movement, which is collected in real time by the IMU and used to realize inertial recursion. Specifically, it includes the drone's three-axis angular velocity data, three-axis specific force data, and timestamps.

[0087] In order to achieve the fusion of multi-source positioning information, RTK-GNSS data, UWB data and IMU data are collected synchronously at the same time interval, that is, each data is collected continuously and has the same timestamp.

[0088] S102. By constructing an inertial recursive prediction model and a GNSS / UWB measurement update model, the optimal state estimates of the UAV based on IMU inertial recursion and GNSS measurement update are obtained respectively.

[0089] The specific implementation steps of step S102 are described below.

[0090] S1021. Construct an inertial recursive prediction model to obtain the inertial recursive state vector of the UAV at time k. The specific construction steps are as follows:

[0091] 1) Obtain the three-axis angular velocities output by the IMU at time k This completes the attitude recursion based on angular velocity, and its expression is:

[0092] ,

[0093] ,

[0094] In the formula, Let be the attitude matrix of the UAV at time k, from the carrier coordinate system to the ENU coordinate system. Let be the attitude matrix of the UAV at time k-1, from the carrier coordinate system to the ENU coordinate system. Let be the attitude change matrix of the UAV from time k−1 to time k. For antisymmetric matrix operators, This refers to the data collection interval.

[0095] 2) Obtain the three-axis ratio output by the IMU at time k. Projecting it onto the ENU coordinate system and subtracting the gravitational acceleration, we obtain the acceleration of the UAV in the navigation coordinate system. Its expression is:

[0096] ,

[0097] In the formula, Let be the gravity vector in the ENU coordinate system. ;

[0098] 3) Construct an inertial recursive prediction model, the expression of which is:

[0099] ,

[0100] ,

[0101] In the formula, For the prior velocity estimate of the UAV at time k, Let be the posterior velocity of the drone at time k-1. Let k be the prior estimated position of the UAV at time k. Let be the posterior position of the UAV at time k-1; where, and All of them come from the previous moment, that is, moment... The calculation results;

[0102] 4) Obtain the inertial recursive state vector of the UAV at time k. Its expression is:

[0103] .

[0104] S1022. Construct a GNSS / UWB measurement update model based on the UAV's inertial recursive state vector at time k. They respectively achieve optimal state estimation based on GNSS data and optimal state estimation based on UWB data.

[0105] 1) Construct a GNSS / UWB measurement update model, the expression of which is:

[0106] ,

[0107] ,

[0108] ,

[0109] ,

[0110] In the formula, Let be the prediction error covariance matrix at time k; Let be the updated error covariance matrix at time k-1; This is the state transition matrix; The process noise covariance matrix; Kalman gain; The noise covariance matrix is ​​set based on the prior accuracy of the sensor. Estimate the optimal state vector of the UAV at time k; Let be the inertial recursive state vector of the UAV at time k; Let be the observation matrix of the UAV at time k; Let be the observation vector of the UAV at time k; Let $k$ be the error covariance matrix of the UAV updated at time $k$.

[0111] Wherein, the state transition matrix process noise covariance matrix Determined using a constant velocity model,

[0112] Process noise covariance matrix The expression is:

[0113] ,

[0114] In the formula, each parameter is pre-adjusted based on the sensor's prior accuracy, actual testing, and statistics;

[0115] State transition matrix The expression is defined as:

[0116] ;

[0117] 2) Based on the UAV's GNSS position and GNSS velocity at time k, construct the UAV's GNSS observation vector at time k. and GNSS observation matrix The expressions for both are:

[0118] ,

[0119] ,

[0120] In the formula, Let k be the coordinates of the UAV's GNSS position on the x-axis, y-axis, and z-axis in the WGS-84 coordinate system at time k. These represent the velocities in the x, y, and z directions of the UAV at time k, calculated based on RTK-GNSS data; where the GNSS observation matrix is... It is a fixed matrix.

[0121] 3) Based on the UWB position of the UAV at time k, construct the UWB observation vector of the UAV at time k. and UWB observation matrix The expressions for both are:

[0122] ,

[0123] ,

[0124] In the formula, Let be the coordinates of the UWB position of the UAV at time k on the x, y, and z axes of the ENU coordinate system; where is the UWB observation matrix. It is a fixed matrix.

[0125] 4) The GNSS observation vector of the UAV at time k constructed in step 2) and GNSS observation matrix Substitute the GNSS / UWB measurement update model constructed in step 1) to obtain the optimal state estimation result of the UAV based on GNSS data at time k. The UWB observation vector of the UAV at time k constructed in step 2) and UWB observation matrix Substitute the GNSS / UWB measurement update model constructed in step 1) to obtain the optimal state estimation result of the UAV based on UWB data at time k. .

[0126] Among them, the optimal state estimation results of the UAV based on GNSS data at time k The state estimation result of the UAV at time k is composed of the velocity and position information of the UAV at time k; while the optimal state estimation result of the UAV at time k based on UWB data is... It consists solely of the speed information of the drone at time k.

[0127] As a preferred technical solution of this embodiment, in order to further eliminate the non-Gaussian outliers caused by multipath effects and high-frequency vibrations, after the above step S102, a sliding window midpoint filter is preferably used to smooth the optimal state estimation results based on GNSS data and the optimal state estimation results based on UWB data acquired by the UAV at continuous time intervals, and the smoothing result is used as the input for subsequent global fusion Kalman filtering.

[0128] In this step, taking time k as an example, the smoothing operation steps for any of the above optimal state estimation results are as follows: Set the sliding window length to N, where N is an odd number; take time k as the center and obtain N consecutive optimal state estimation results from the past (N-1) / 2 sampling times to the future (N-1) / 2 sampling times; sort the N consecutive optimal state estimation results and take the median value as the effective observation value at the current time k, so as to update the optimal state estimation results based on GNSS data and the optimal state estimation results based on UWB data at time k of the UAV. The sliding window length N ranges from 5 to 11; in this embodiment, N=7.

[0129] In practice, for N optimal state estimation results based on GNSS data, the combination of the median values ​​of the N GNSS positions and the median values ​​of the N GNSS velocities is used as the optimal state estimation result based on GNSS data at time k of the UAV, replacing the initial optimal state estimation result; for N optimal state estimation results based on UWB data, the median value of the N UWB positions is used as the optimal state estimation result based on UWB data at time k of the UAV, replacing the initial optimal state estimation result; the above two parts of optimal state estimation results are used as measurement inputs for the global filter in the subsequent process.

[0130] S103. Transform the optimal state estimation result based on GNSS data at time k of the UAV from the WGS-84 coordinate system (the globally universal geocentric coordinate system) to the ENU coordinate system (northeast-east-sky coordinate system) to unify the spatial reference system.

[0131] The specific implementation steps of step S103 are as follows:

[0132] S1031. Obtain the optimal state estimation result of the UAV based on GNSS data at time k, and determine its longitude. ,latitude and elevation First, transform from the WGS-84 coordinate system to the ECEF coordinate system (Earth-centered Earth-fixed coordinate system) in three-dimensional coordinates. Its conversion expression is:

[0133] ,

[0134] ,

[0135] ,

[0136] In the formula, The square of the first eccentricity, , , , , Let be the radius of curvature of the circle. .

[0137] S1032. Select the coordinates of the static takeoff point or the benchmark point of the survey area. As the origin of the ENU coordinate system Through rotation matrix Project the three-dimensional coordinates of the UAV at time k in the ECEF coordinate system onto the ENU coordinate system to obtain the observation position of the UAV at time k in the ENU coordinate system. Its conversion expression is:

[0138] ,

[0139] .

[0140] S104. Construct a global optimal state estimation model, and use the optimal state estimation results based on GNSS data and the optimal state estimation results based on UWB data at time k of the UAV under the unified ENU coordinate system to obtain the global optimal state estimation of the UAV at time k in the ENU coordinate system.

[0141] The specific implementation steps of step S104 are described below.

[0142] S1041. Obtain the optimal state estimation results based on GNSS data and UWB data for the UAV at time k in the unified ENU coordinate system, and construct the global observation vector for the UAV at time k. Its expression is:

[0143] ,

[0144] In the formula, The coordinates of the observation position of the UAV in the ENU coordinate system at time k are respectively on the x-axis, y-axis and z-axis, which are determined by step S103; The optimal state estimation result of the UAV based on UWB data at time k is the position estimation coordinates on the x-axis, y-axis and z-axis, which is specifically determined by step S102. These are the velocity components in the x, y, and z directions of the optimal state estimation result based on GNSS data at time k for the UAV, specifically determined in step S102. It should be noted that since the GNSS receiver can directly output the velocity in the ENU coordinate system, therefore... It can be obtained directly.

[0145] S1042. Since the UAV's state variables and observation variables are in the same ENU coordinate system and physical dimensions, the observation equation exhibits a strictly linear mapping relationship; based on this, the UAV's global observation matrix at time k... Designed as follows:

[0146] ;

[0147] Correspondingly, the global observation vector of the UAV at time k Determined by step S1041;

[0148] S1043. Construct a globally optimal state estimation model, the expression of which is:

[0149] ,

[0150] ,

[0151] ,

[0152] ,

[0153] ,

[0154] In the formula, Let be the global prediction error covariance matrix at time k; The global error covariance matrix is ​​updated at time k-1; This is the state transition matrix; The process noise covariance matrix; For global Kalman gain; The measurement noise covariance matrix; This is the global optimal state vector estimate for the UAV at time k; Let be the global inertial recursive state vector of the UAV at time k; Let be the global observation matrix of the UAV at time k; Let be the global observation vector of the UAV at time k; Let $k$ be the global error covariance matrix of the UAV updated at time $k$.

[0155] Wherein, the state transition matrix process noise covariance matrix Determined using a constant velocity model,

[0156] Process noise covariance matrix The expression is:

[0157] ,

[0158] In the formula, each parameter is pre-adjusted based on the sensor's prior accuracy, actual testing, and statistics;

[0159] State transition matrix The expression is defined as:

[0160] ;

[0161] S1044. The global inertial recursive state vector of the UAV at time k, determined by steps S1041-S1043. The global observation vector of the UAV at time k and the global observation matrix of the UAV at time k. Substituting these values ​​into the global optimal state estimation model, we can obtain the global optimal state vector estimate of the UAV at time k. The global error covariance matrix of the UAV at time k. .

[0162] Similarly, the global optimal state vector estimation of the UAV at time k Specifically, it consists of the position and velocity of the UAV in the ENU coordinate system at time k, and its expression is:

[0163] .

[0164] S105. Estimate the global optimal state vector of the UAV at time k, which was output in step S104. Inversely convert the position components in the coordinate system to longitude and latitude in the WGS-84 coordinate system. And elevation.

[0165] Specifically, the implementation steps of step S106 are described as follows.

[0166] S1051, Using a rotation matrix The global optimal state vector of the UAV at time k is estimated. The position in the middle To obtain its coordinate increment in the ECEF coordinate system Its expression is:

[0167] ;

[0168] S1052. Add the ECEF coordinate increment obtained in step S1051 to the coordinates of the ENU coordinate system origin in the ECEF coordinate system to obtain the absolute coordinates of the UAV's position in the ECEF coordinate system at time k. Its calculation expression is:

[0169] ;

[0170] S1053. Calculate intermediate variables, including the projected length of the ECEF coordinates onto the equatorial plane. and auxiliary angle ,in,

[0171] The length of the ECEF coordinates projected onto the equatorial plane The calculation expression is:

[0172] ;

[0173] auxiliary angle The calculation expression is:

[0174] ,

[0175] In the formula, The semi-major axis of the WGS-84 ellipsoid. ; For the minor semi-axis of the ellipsoid, For the flattening of the ellipsoid, ; It is the arctangent function in the four quadrants.

[0176] S1054. Using Bowring's single approximation formula, the latitude is calculated sequentially. ,longitude and elevation Its expression is:

[0177] ,

[0178] ;

[0179] ;

[0180] In the formula, The square of the first eccentricity, The square of the second eccentricity, As an auxiliary angle;

[0181] S1055. Based on the pre-calibrated system offset, calculate the latitude obtained in step S1064. ,longitude and elevation Perform system deviation correction and convert to degree system output.

[0182] Specifically, the specific operation steps of step S1055 are as follows:

[0183] 1) Calculate the latitude and longitude Subtract the pre-calibrated system offset to obtain the latitude offset correction. Longitude offset correction Both are measured in radians.

[0184] 2) Adjust the latitude offset Longitude offset correction The expression for converting radians to degrees is:

[0185] ,

[0186] ,

[0187] 3) Obtain the final fused position result of the UAV at the current time (i.e., time k). In this system, longitude comes first, latitude comes second, and elevation is in meters.

[0188] In step 1) above, the pre-calibrated system offset specifically refers to the difference between the sensor output and the actual real position data. Based on this, the system offset can be calculated by placing the UAV on one or more reference points with known precise coordinates, collecting the positioning data output by the system over a long period of time, and calculating the average deviation between the output value and the real value, which is regarded as the system offset.

[0189] S2. Construct a road defect identification network model to process the continuous video frames collected during the UAV inspection process frame by frame, identify the road defects in each frame, and output the image pixel coordinates and road defect detection boxes containing the road defect categories.

[0190] The specific implementation steps of step S2 are described below.

[0191] S201. Based on the improvement of the YOLOv8 network, a road defect identification network model is constructed.

[0192] 1) Construct a flexible attention module (FA) to suppress background noise and enhance disease features through channel attention mechanism.

[0193] Specifically, this flexible attention module first uses global average pooling to obtain channel-wide feature descriptors, and then generates an adaptive weight matrix through two fully connected layers and an activation function. Its core module consists of a channel attention-feature enhancement chain, and its expression is as follows:

[0194] ,

[0195] ,

[0196] ,

[0197] ,

[0198] ,

[0199] ,

[0200] In the formula, This represents the input feature map, with dimensions... ; Indicates the feature map height; Indicates the width of the feature map; Indicates the number of channels in the feature map; This represents an adaptive global average pooling function used to compress spatial dimensions to... Extract global features from each channel; This represents the feature vector after global average pooling, with dimension [missing information]. ; This represents the result after feature compression, with dimensions... ; This represents the compression ratio, used to reduce the number of computational parameters. In this embodiment... 4; This is the weight matrix of the first fully connected layer, with dimensions [missing information]. ; This represents the weight matrix of the second fully connected layer, with dimensions [not specified]. ; This represents the result after feature recovery; This represents the channel attention weight matrix, with dimensions... ,range ; The sigmoid activation function is used to map numerical values ​​to... interval; This represents the feature map after attention enhancement; Hadamard product; ReLU is the modified linear unit activation function, used to introduce nonlinear features; This represents the features after ReLU activation, in terms of dimension. .

[0201] 2) Construct a spatial downsampling module (SD) to optimize airborne end-side computational overhead through convolutional downsampling and batch normalization.

[0202] This spatial downsampling module passes the input feature map through a step size of 2. Convolution performs spatial downsampling and uses channel attention for reconstruction, as expressed in the following expression:

[0203] ,

[0204] ,

[0205] ,

[0206] In the formula, Indicates the input feature map; Indicates the kernel size as Downsampling convolutional layers with a stride of 2; This indicates a batch normalization layer, used to accelerate model convergence and stabilize the training process; This represents the channel attention weight matrix. It is the sigmoid activation function. This is the final output of an efficient downsampling feature map.

[0207] 3) Improvements to the YOLOv8 network based on the flexible attention module constructed in step 1) and the spatial downsampling module constructed in step 2), including:

[0208] Replace the convolutional modules in layers 0, 1, 3, 5, and 7 of the backbone network with spatial downsampling modules (SD), and add flexible attention modules (FA) at the output of the newly replaced spatial downsampling modules (SD) in layers 3, 5, and 7.

[0209] Replace the convolutional modules in layers 16 and 19 of the head network with spatial downsampling (SD) modules, and add flexible attention (FA) modules to the output of the newly replaced spatial downsampling (SD) modules in layers 16 and 19.

[0210] S202. Construct a road defect image dataset for training a road defect recognition network model;

[0211] Specifically, the road defect image dataset was constructed by using drones to collect a large number of images containing different types of road defects, marking the road defects in each image with detection boxes and setting road defect category labels.

[0212] S203. The road defect image dataset constructed in step S202 is used to train the road defect recognition network model constructed in step S201, so that after inputting road images collected by UAV, the road defect recognition network model can output image pixel coordinates containing road defect categories and road defect detection boxes.

[0213] S3. Construct an inverse perspective mapping model to combine the UAV's 3D position and attitude, camera offset, and camera intrinsic parameters obtained in step S1 to convert the image pixel coordinates of the road defects obtained in step S2 into the 3D physical coordinates of the road defects in the real world coordinate system.

[0214] The following describes the specific steps for obtaining the three-dimensional physical coordinates of road defects in the real-world coordinate system in step S3, taking any road defect detection box in any image result output by the road defect identification network model in step S2 as an example.

[0215] S301. Extract the coordinates of the center point of the road defect detection frame. , and serve as the initial position of the road defect in the image pixel coordinate system;

[0216] S302. Obtain the absolute position of the optical center of the UAV-borne camera used for acquiring road inspection images in the ENU coordinate system. With absolute attitude rotation matrix The specific operational steps for understanding the relationship between them are as follows:

[0217] S3021. Obtain the fused pose of the UAV at time k in the ENU coordinate system, which is determined by its position. and attitude quaternions Composition; among which, position Specifically, substituting the fused position result of the UAV at time k obtained from step S1, the attitude quaternion... The attitude matrix of the UAV at time k from the carrier coordinate system to the ENU coordinate system in step S1 get;

[0218] attitude quaternions Convert to standard Direction cosine matrix Its expression is:

[0219] ;

[0220] S3022. Determine the translation offset vector of the camera optical center relative to the UAV body coordinate system as follows: And camera Euler angle offset ,but:

[0221] Based on the ZYX rotation order, the camera Euler angle offset is converted into a relative rotation matrix. :

[0222] ,

[0223] Furthermore, based on the rigid body kinematics equations, the absolute position of the camera's optical center in the ENU coordinate system is obtained. With absolute attitude rotation matrix The expression for the relationship between them is:

[0224] ,

[0225] ,

[0226] S303. Calculate the unit direction vector of the projection ray at the pixel coordinates;

[0227] Based on the pinhole camera model, the camera image resolution is... ( Image width, (Image height) is used to calculate the camera's physical equivalent focal length. The unit is pixels, and its calculation expression is:

[0228] ,

[0229] In the formula, Image width, This refers to the horizontal field of view.

[0230] With the center point of the image Using the origin as the initial point, the road defects are located in the image pixel coordinate system at their initial position (pixels). Represented as the ray direction vector in the camera coordinate system Its expression is:

[0231] ;

[0232] Furthermore, regarding the ray direction vector After normalizing the magnitude, the unit direction vector is obtained. The expression for modulus normalization is: .

[0233] S304. By intersecting the spatial ray with the road surface plane, the precise three-dimensional physical coordinates of the road defect at that location can be obtained. .

[0234] The camera ray unit direction vector obtained in step S3023 Unit direction vector transformed to ENU coordinate system Its conversion expression is:

[0235] ;

[0236] In a highway inspection scenario, a local area of ​​the road surface is approximated as a horizontal plane, and the absolute elevation of this road surface in the ENU coordinate system is obtained. ;

[0237] Using the ray parameter equations, the solution obtained is the image with respect to the camera's optical center. Starting point A spatial ray with direction and a plane spatial intersection ;

[0238] The expression for the ray parameter equation is as follows:

[0239] ,

[0240] The scaling parameter is obtained by expanding... The axis components are obtained by solving. ;

[0241] In practical applications, if and only if the scaling parameter When the ray intersects the ground, determine the point of intersection; adjust the scaling parameter. Substituting back into the ray parameter equations, the spatial intersection points can be obtained. That is, the precise three-dimensional physical coordinates of the road damage in the real-world coordinate system. .

[0242] And when the proportional parameter If the ray does not physically intersect the road surface, it means that the drone's onboard camera is pointing towards the sky, rather than inspecting the road.

[0243] S4. Using the sliding window majority voting logic, the three-dimensional physical coordinates of road defects mapped in the continuous frame video stream are effectively confirmed, and road defects belonging to the same source are spatially merged and clustered to eliminate false alarms and redundant detection results, and output the valid three-dimensional physical coordinates of road defects.

[0244] Specifically, the implementation steps of step S4 are described as follows.

[0245] S401. Define the sliding window majority voting logic, specifically including: setting the time window length. The images processed in step S3 are randomly sampled sequentially within each time window; the maximum number of sampling frames within a single window is set. To determine the number of image frames randomly selected within each time window length. Set the minimum confidence threshold. This serves as the criterion for judging the existence of road defects, specifically the number of frames in the extracted images within a time window that detect the same type of road defect. If so, the road defect is determined to be a validly detected road defect. .

[0246] In this embodiment, the time window length Set to 0.5s, maximum number of sampling frames per window For 3 frames, the lowest confidence threshold It consists of 2 frames.

[0247] S402. Using spatial distance clustering, the set of valid disease points confirmed in step S401 is processed. The physical entities of road defects are spatially deduplicated, among which... This represents the i-th road defect point, which is represented by the three-dimensional physical coordinates of the road defect. and road disease attributes constitute, , This represents the kth type of road defect attribute (including but not limited to potholes, transverse cracks, longitudinal cracks, etc.); This represents the total number of road defects.

[0248] Specifically, the operation steps of step S402 are described as follows.

[0249] S4021, Collection of effective disease points All road defects are classified according to their attributes. Grouping;

[0250] S4022. For all road defect points in the same group, calculate the Euclidean distance between every two road defect points sequentially, and then apply the calculated distance based on the preset spatial clustering distance threshold. Determine whether they belong to the same source of road defects;

[0251] For any road defect point in the same group and The Euclidean distance between the two The calculation expression is:

[0252] ,

[0253] In the formula, and Road defects Three-dimensional physical coordinates The X and Y coordinates are in the middle. and Road defects Three-dimensional physical coordinates The X and Y coordinates are in the middle.

[0254] Furthermore, if If the two road defects are found to be of the same origin, then they are determined to be road defects from the same source; otherwise, they are determined to be road defects from different origins.

[0255] S403. Multiple road disease points identified as belonging to the same source as determined in step S402 are merged into one, and the moving average method is used to update and calculate the three-dimensional physical coordinates of the multiple road disease points, so as to finally output the three-dimensional physical coordinates of the effective road disease and its corresponding category attributes.

[0256] In practical applications, such as Figure 1 As shown, based on the results obtained from the above inspection method, the positioning data of the UAV and the processed three-dimensional coordinates of road defects can be pushed to the cloud server in parallel using a wireless communication network, and then pushed to the user's visualization terminal to realize real-time UAV trajectory rendering and road defect early warning.

[0257] In practice, the aforementioned real-time drone trajectory rendering and road defect early warning can utilize the WebSocket protocol to report the processed and simplified final JSON data packet (containing coordinates and road defect categories) to the public cloud; the cloud server then uses WebSocket to push the stream to the user terminal via the intranet. The front-end dynamically renders the drone coordinates, defect coordinates, and defect details on the mobile device, completing the final closed-loop real-time management, maintenance, and early warning system.

[0258] Furthermore, the correctness and effectiveness of the method of the present invention are verified through different experiments.

[0259] Experimental verification of the road defect identification network model constructed in step S2; specifically, using four sets of experiments, the impact of different improved modules on the performance of road defect detection is compared.

[0260] The specific test results are shown in Table 1.

[0261] Table 1:

[0262] Original YOLOv8 network 68% 14.2ms 0.75 0.65 6.8M The FA module is added only at the output of the specified convolutional layer; the DS module is not replaced. 72% 14.5ms 0.78 0.68 7.2M The DS module was replaced only on the specified convolutional layer; the FA module was not added to the output of the DS module. 75% 14.8ms 0.80 0.70 7.5M Method of the present invention 89% 15.4ms 0.84 0.84 8.1M

[0263] As can be seen from the comparison in Table 1, the complete improved model, supported by the efficient downsampling of the SD module, achieves an mAP@0.5 of 89% and an inference time of only 15.4ms, realizing a significant improvement in accuracy at a low cost. Therefore, the effectiveness and correctness of the method provided in this invention are verified.

[0264] Furthermore, comparative experiments were also conducted to verify the effectiveness of step S4 of the present invention. Specifically, four sets of control experiments were repeated, including: an experiment without filtering (i.e., directly outputting the three-dimensional physical coordinates of road defects and their corresponding category attributes based on the results of step S3), an experiment with only time window filtering (i.e., outputting the three-dimensional physical coordinates of road defects and their corresponding category attributes only after step S401), and an experiment with only spatial clustering filtering (i.e., outputting the three-dimensional physical coordinates of road defects and their corresponding category attributes only after steps S402 and S403) as control examples.

[0265] In the four sets of comparative experiments, such as Figure 2 The figure shows a performance comparison of different filtering methods before and after optimization using the method of this application, focusing on the rejection rate and false positive rate; Figure 3The figure shown is a performance comparison chart of different filtering methods before and after optimization using the method of this application in terms of the number of output defects.

[0266] from Figure 2 The performance comparison results show that the method of this invention achieves a rejection rate of up to 97.2%, while reducing the false positive rate to 0%; from Figure 3 The performance comparison results show that after spatiotemporal filtering, the original approximately 430 detections finally converged into 12 high-confidence real disease targets. This significant difference proves the strong suppression ability of the present invention against random noise and environmental artifacts (such as shadows and reflections), and can greatly reduce data redundancy, thus confirming the effectiveness and correctness of the method provided by the present invention.

[0267] The parts of this invention not disclosed in detail are well-known in the art. Although illustrative specific embodiments of the invention have been described above to help those skilled in the art understand the invention, it should be understood that the invention is not limited to the scope of the specific embodiments, and various modifications will be readily apparent to those skilled in the art as long as they are within the limits of the appended claims.

Claims

1. A method for intelligent highway inspection using unmanned aerial vehicles (UAVs) based on multi-source positioning information fusion, characterized in that, The steps are as follows: S1. Utilize UAV-borne multi-source sensors to synchronously acquire RTK-GNSS data, UWB data, and IMU data. First, construct an inertial recursive prediction model and a GNSS / UWB measurement update model. Based on IMU inertial recursion, obtain the optimal state estimation results based on GNSS data and the optimal state estimation results based on UWB data, respectively, under the same spatial reference frame. Construct a global optimal state estimation model and use the optimal state estimation results based on GNSS data and the optimal state estimation results based on UWB data at time k under the unified ENU coordinate system to obtain the global optimal state estimation of the UAV at time k in the ENU coordinate system. S2. Construct a road defect identification network model to process the continuous video frames collected during the UAV inspection process frame by frame, identify the road defects in each frame, and output the image pixel coordinates and road defect detection boxes containing the road defect categories. S3. Construct an inverse perspective mapping model to combine the fusion position result of the UAV time k obtained in step S1, the camera offset, and the camera intrinsic parameters to convert the image pixel coordinates of the road defects obtained in step S2 into the three-dimensional physical coordinates of the road defects in the real world coordinate system. S4. Using the sliding window majority voting logic, the three-dimensional physical coordinates of road defects mapped in the continuous frame video stream are effectively confirmed, and road defects belonging to the same source are spatially merged and clustered to output the effective three-dimensional physical coordinates of road defects.

2. The method for intelligent highway inspection using unmanned aerial vehicles (UAVs) based on multi-source positioning information fusion according to claim 1, characterized in that, In step S1, the method for obtaining the optimal state estimation results based on GNSS data and the optimal state estimation results based on UWB data at time k of the UAV is as follows: 1) Construct an inertial recursive prediction model to obtain the inertial recursive state vector of the UAV at time k. ; The expression for the inertial recursive prediction model is: ， ， In the formula, For the prior velocity estimate of the UAV at time k, Let be the posterior velocity of the drone at time k-1. Let k be the prior estimated position of the UAV at time k. Let k be the posterior position of the drone at time k-1; in, The acceleration of the UAV in the navigation coordinate system is expressed as follows: , Let be the attitude matrix of the UAV at time k, from the carrier coordinate system to the ENU coordinate system, and its expression is: , , The three-axis angular velocities output by the IMU at time k. This refers to the data collection interval. For antisymmetric matrix operators; Let be the gravity vector in the ENU coordinate system. ; The three-axis specific force output by the IMU at time k; Furthermore, the inertial recursive state vector of the UAV at time k is obtained. : ; 2) Construct a GNSS / UWB measurement update model based on the UAV's inertial recursive state vector at time k. The optimal state estimates based on GNSS data and the optimal state estimates based on UWB data were obtained respectively. The expression for the GNSS / UWB measurement update model is: ， ， ， ， In the formula, Let be the prediction error covariance matrix at time k; Let be the updated error covariance matrix at time k-1; This is the state transition matrix; The process noise covariance matrix; Kalman gain; The noise covariance matrix is ​​set based on the prior accuracy of the sensor. Estimate the optimal state vector of the UAV at time k; Let be the inertial recursive state vector of the UAV at time k; Let be the observation matrix of the UAV at time k; Let be the observation vector of the UAV at time k; Let $k$ be the error covariance matrix of the UAV updated at time $k$. 3) Construct the GNSS observation vector of the UAV at time k and GNSS observation matrix The data is then substituted into the GNSS / UWB measurement update model to obtain the optimal state estimate of the UAV based on GNSS data at time k. Construct the UWB observation vector of the UAV at time k. and UWB observation matrix The data is then substituted into the GNSS / UWB measurement update model to obtain the optimal state estimate of the UAV based on UWB data at time k. .

3. The method for intelligent highway inspection using unmanned aerial vehicles (UAVs) based on multi-source positioning information fusion according to claim 2, characterized in that, GNSS observation vector of UAV at time k and GNSS observation matrix The GNSS position and GNSS velocity of the UAV at time k are constructed, and their expressions are as follows: ， ， In the formula, Let k be the coordinates of the UAV's GNSS position on the x-axis, y-axis, and z-axis in the WGS-84 coordinate system at time k. These represent the velocities in the x, y, and z directions of the UAV at time k, calculated based on RTK-GNSS data. UWB observation vector of the UAV at time k and UWB observation matrix The location is constructed based on the UWB position of the UAV at time k, and the expressions for both are: ， ， In the formula, Let k be the coordinates of the UWB position of the UAV at time k on the x-axis, y-axis, and z-axis of the ENU coordinate system.

4. The method for intelligent highway inspection using unmanned aerial vehicles (UAVs) based on multi-source positioning information fusion according to claim 1, characterized in that, In step S1, the optimal state estimation results based on GNSS data acquired by the UAV at consecutive time points are evaluated using a sliding window mid-value filter. And the optimal state estimation results based on UWB data. Smoothing is performed; the smoothing method for any optimal state estimation result of the UAV at time k is as follows: set the sliding window length to N, where N is an odd number; take time k as the center, obtain N consecutive optimal state estimation results from the past (N-1) / 2 sampling times to the future (N-1) / 2 sampling times; sort the N consecutive optimal state estimation results, and take the median value as the effective observation value of the current time k, so as to update the optimal state estimation result based on GNSS data and the optimal state estimation result based on UWB data at time k of the UAV.

5. The method for intelligent highway inspection using unmanned aerial vehicles (UAVs) based on multi-source positioning information fusion according to claim 1, characterized in that, In step S1, the method for obtaining the global optimal state estimate of the UAV at time k in the ENU coordinate system is as follows: 1) Transform the optimal state estimation results based on GNSS data at time k of the UAV from the WGS-84 coordinate system to the ENU coordinate system; 2) Based on the optimal state estimation results of the UAV at time k in the unified ENU coordinate system using GNSS data and UWB data, a global observation vector for UAV at time k is constructed. Correspondingly, the global observation matrix of the UAV at time k Designed as follows: ； 3) Construct a globally optimal state estimation model, the expression of which is: ， ， ， ， ， In the formula, Let be the global prediction error covariance matrix at time k; The global error covariance matrix is ​​updated at time k-1; This is the state transition matrix; The process noise covariance matrix; For global Kalman gain; The measurement noise covariance matrix; This is the global optimal state vector estimate for the UAV at time k; Let be the global inertial recursive state vector of the UAV at time k; Let be the global observation matrix of the UAV at time k; Let be the global observation vector of the UAV at time k; Let $k$ be the global error covariance matrix of the UAV updated at time $k$. 4) Solve to obtain the global optimal state vector estimate of the UAV at time k. The global error covariance matrix of the UAV at time k. .

6. The method for intelligent highway inspection using unmanned aerial vehicles (UAVs) based on multi-source positioning information fusion according to claim 1, characterized in that, In step S2, the road defect identification network model is constructed based on an improved YOLOv8 network. The specific construction steps are as follows: 1) Construct a flexible attention module, which uses global average pooling to obtain channel global feature descriptors, and then generates an adaptive weight matrix through two fully connected networks and activation functions. Finally, the adaptive weight matrix is ​​processed by performing a Hadamard product with the input feature map to output the attention-enhanced feature map. 2) Construct a spatial downsampling module, which takes the input feature map and passes it through a step size of 2. Convolution performs spatial downsampling and utilizes channel attention for recombination to output an efficient downsampled feature map; 3) Improvements to the YOLOv8 network include: replacing the convolutional modules in layers 0, 1, 3, 5, and 7 of the backbone network with spatial downsampling modules, and adding flexible attention modules at the outputs of the newly replaced spatial downsampling modules in layers 3, 5, and 7; replacing the convolutional modules in layers 16 and 19 of the head network with spatial downsampling modules, and adding flexible attention modules at the outputs of the newly replaced spatial downsampling modules in layers 16 and 19.

7. The method for intelligent highway inspection using unmanned aerial vehicles (UAVs) based on multi-source positioning information fusion according to claim 5, characterized in that, In step S3, the specific method for converting any road defect detection box in any image result output by the road defect recognition network model in step S2 is as follows: S301. Extract the coordinates of the center point of the road defect detection frame. And this serves as the initial position of the road defect in the image pixel coordinate system; S302. Obtain the fused pose of the UAV at time k in the ENU coordinate system, which is determined by its position. and attitude quaternions Composition, attitude quaternions Convert to standard Direction cosine matrix The translation offset vector of the camera's optical center relative to the UAV's body coordinate system is determined as follows: And the camera Euler angle bias, and convert the camera Euler angle bias into a relative rotation matrix. Then, using the rigid body kinematics equations, the absolute position of the camera's optical center in the ENU coordinate system can be obtained. With absolute attitude rotation matrix The expression for the relationship between them is: ， ； Wherein, the fused pose of the UAV at time k in the ENU coordinate system is the estimated global optimal state vector of the UAV at time k obtained in step S1. The position components in the image are inversely transformed to their positions in the WGS-84 coordinate system. S303, Based on the width of the image captured by the camera and horizontal field of view Calculate the camera's physical equivalent focal length. : ; with the center point of the image Using the origin as the initial point, the road defects are located in the image pixel coordinate system at their initial position (pixels). Represented as the ray direction vector in the camera coordinate system : ; for the ray direction vector After normalizing the magnitude, the unit direction vector is obtained. : ; S304. The camera ray unit direction vector obtained from step S3023 is... Unit direction vector transformed to ENU coordinate system : In a highway inspection scenario, a local area of ​​the road surface is approximated as a horizontal plane, and the absolute elevation of this road surface in the ENU coordinate system is obtained. Using the ray parameter equation: Solving for the optical center of the camera yields the solution. Starting point A spatial ray with direction and a plane spatial intersection , The scaling parameter is obtained by expanding... The axis components are obtained by solving. , and The absolute elevations are respectively and the unit direction vector of the camera ray The component on the Z-axis; if and only if the scaling parameter When, find the spatial intersection point. That is, to obtain the three-dimensional physical coordinates of road defects in the real-world coordinate system. .

8. The method for intelligent highway inspection using unmanned aerial vehicles (UAVs) based on multi-source positioning information fusion according to claim 7, characterized in that, In step S302, the fused pose of the UAV at time k in the ENU coordinate system is the estimated global optimal state vector of the UAV at time k obtained in step S1. The position components in the coordinate system are inversely transformed to longitude, latitude, and elevation in the WGS-84 coordinate system.

9. The method for intelligent highway inspection using unmanned aerial vehicles (UAVs) based on multi-source positioning information fusion according to claim 6, characterized in that, The specific implementation steps of step S4 are as follows: S401. Define the sliding window majority voting logic, including: setting the time window length. The images processed in step S3 are randomly sampled sequentially within each time window; the maximum number of sampling frames within a single window is set. To determine the number of image frames randomly selected within each time window length. Set the minimum confidence threshold. This serves as the basis for judging the existence of road defects. The number of frames in the extracted images within the time window that detect the same type of road defect is considered. If so, the road defect is determined to be a validly detected road defect. ; S402. Using spatial distance clustering, the set of valid disease points confirmed in step S401 is processed. The physical entities of road defects are spatially deduplicated, among which... This represents the i-th road defect point, which is determined by the three-dimensional physical coordinates of the road defect. and road disease attributes constitute, , This represents the attribute of the k-th type of road defect; This represents the total number of road defects; S403. Multiple road disease points identified as belonging to the same source as determined in step S402 are merged into one, and the moving average method is used to update and calculate the three-dimensional physical coordinates of the multiple road disease points to output the three-dimensional physical coordinates of the effective road disease and its corresponding category attributes.

10. The method for intelligent highway inspection by unmanned aerial vehicle (UAV) based on multi-source positioning information fusion according to claim 9, characterized in that, The specific operation steps for step S402 are as follows: S4021, Collection of effective disease points All road defects are classified according to their attributes. Grouping; S4022. For all road defect points in the same group, calculate the Euclidean distance between every two road defect points sequentially, and then apply the calculated distance based on the preset spatial clustering distance threshold. Determine whether they belong to the same source of road defects; For any road defect point in the same group and The Euclidean distance between the two The calculation expression is: ， In the formula, and Road defects Three-dimensional physical coordinates The X and Y coordinates are in the middle. and Road defects Three-dimensional physical coordinates The X and Y coordinates are in the middle. Furthermore, if If the two road defects are found to be of the same origin, then they are determined to be road defects from the same source; otherwise, they are determined to be road defects from different origins.

Images