Bridge Pier Measurement Method and System Based on the Fusion of Laser Gyroscopic Measuring Rod and UAV Vision

Abstract

The present application discloses a pier measurement method and system that integrates a laser gyro measurement rod with UAV vision. The method includes: a UAV carries a laser gyro measurement rod integrated with a gyroscope, an inclination sensor, and a wireless communication module, flies to the pier measurement area, and uses the laser gyro measurement rod to collect the three-dimensional attitude data and inertial reference data of the UAV in real time; the UAV synchronously activates a binocular vision module and a laser rangefinder, and performs multi-view scanning of the pier along a preset surrounding path to obtain the surface feature point cloud and real-time distance data of the pier; and aligns the inertial reference data, the vision point cloud, and the real-time distance data in space and time. The present invention solves the problem of serious distortion of measurement data caused by perception degradation under complex meteorological conditions through the space-time fusion of laser gyro-vision point cloud, GAN-enhanced anti-rain and fog verification, and dynamic anti-wind closed-loop control, so as to accurately extract the true geometric features of the pier.

Description

Technical Field

[0001] This application relates to the technical field of pier measurement, and particularly to a pier measurement method and system that integrates a laser gyro measurement rod and UAV vision. Background Art

[0002] In the field of bridge structure detection, the UAV pier measurement technology has long faced the problem of data reliability under the interference of dynamic environments, and the existing technology has proposed a dynamic perception method that combines UAV vision with vision.

[0003] For example, Chinese Patent No. CN116182837A proposes a positioning and mapping method based on the tight coupling of visual lidar and inertia to obtain high-precision positioning trajectories and geometric structure information of the surrounding environment, and can maintain good robustness and accuracy in typical lidar degradation scenarios. This invention, the positioning and mapping method based on the tight coupling of visual lidar and inertia, first processes the data of three sensors: autonomous positioning and mapping, lidar autonomous positioning and mapping, and vision and inertial measurement unit IMU respectively; secondly, constructs visual reprojection, IMU pre-integration and radar geometric constraints, and jointly establishes a non-linear optimization problem with three constraints; finally, uses the optimized pose and the current frame radar point cloud to update the voxel map, providing geometric structure information for subsequent processes. This invention is mainly applied to the occasions of UAV design and manufacturing.

[0004] However, when the above technology is applied to pier measurement, the coupling superposition of strong wind disturbance and the motion noise of the UAV itself will occur, resulting in serious distortion of the measurement data, making it difficult to accurately extract the true geometric features of the pier. This core problem further causes the problem of perception degradation under complex meteorological conditions - the visual features are blurred in environments such as rain, fog, and low light, and traditional detection methods are forced to rely on a single sensor, resulting in a significant decrease in measurement robustness. Most of the existing technologies alleviate the interference by restricting flight conditions or increasing hardware redundancy, but there are defects such as low efficiency and limited adaptability. Summary of the Invention

[0005] To solve the above problems, an embodiment of the present invention provides a pier measurement method that integrates a laser gyro measurement rod and UAV vision. The method includes:

[0006] A UAV is equipped with a laser gyro measurement rod integrated with a gyroscope, an inclination sensor, and a wireless communication module, and flies to the pier measurement area, and the three-dimensional attitude data and inertial reference data of the UAV are collected in real time through the laser gyro measurement rod.

[0007] The UAV synchronously starts the binocular vision module and the laser rangefinder, and performs multi-view scanning on the pier along a preset surrounding path to obtain the surface feature point cloud and real-time distance data of the pier.

[0008] Perform spatio-temporal alignment on inertial reference data, visual point cloud, and real-time distance data;

[0009] Adopt an anti-interference extended Kalman filter algorithm to fuse multi-source data, separate the actual size data of the bridge pier from the UAV motion noise, and generate the three-dimensional actual size data of the bridge pier and the heat map of size deviation;

[0010] According to the confidence level of the heat map of size deviation, dynamically plan the local refined scanning path of the UAV, and avoid obstacles in real time through the laser rangefinder until the measurement accuracy meets the standard.

[0011] Furthermore, the binocular vision module calls an image enhancement model based on the generative adversarial network in rainy and foggy environments to remove fog and repair textures for low visibility images.

[0012] Furthermore, the spatio-temporal alignment method includes:

[0013] Compensate for the pose jitter of the UAV itself based on the high-frequency attitude data of the gyroscope;

[0014] Dynamically bind the camera coordinate system of the binocular vision module, the laser ranging coordinate system, and the inertial reference coordinate system of the laser gyro measurement rod;

[0015] During the hovering stage of the UAV, use the static data of the laser gyro measurement rod to perform online zero bias calibration on the binocular vision module and the laser rangefinder.

[0016] Furthermore, the method of dynamic binding includes:

[0017] Based on the inertial data of the laser gyro measurement rod, forcefully correct the pose estimation result of visual SLAM;

[0018] The indicators of visual SLAM include the number of feature point tracks, reprojection error, and the trace of the pose estimation covariance matrix. If any indicator exceeds the limit, it is determined that visual SLAM fails and forced correction is triggered;

[0019] When visual SLAM fails, based on the pose of the gyroscope, forcefully correct the output of visual SLAM through extended Kalman filtering. When forcefully correcting, the observation value is switched to the inertial pose, and the observation noise covariance is increased to reflect the drift characteristics of the inertial data. At the same time, filter update is performed. The filter update equation includes:

[0020] ;

[0021] In the formula, is the posterior state estimate; is the prior state estimate; is the Kalman gain; is the observation vector; is the observation matrix.

[0022] Further, the generation of the three-dimensional actual size data and the size deviation heat map of the pier includes:

[0023] The drone flies along the axial direction of the pier at a constant distance and scans the surface at a fixed frequency;

[0024] The IMU data is aligned with the time stamp of the laser point cloud, and the visual image is used to assist in coloring the point cloud;

[0025] Initialize the state covariance matrix of the filter and set the prior distribution of the motion noise; regard the actual size parameters of the pier and the motion noise of the drone as the joint state variables;

[0026] Preprocess the scanned data, introduce an adaptive noise covariance matrix, and dynamically downweight the abnormal jump points;

[0027] Iteratively execute the prediction-update step, decouple the static size data of the pier and the dynamic motion noise of the drone through the observation model, and output the denoised three-dimensional actual size data of the pier;

[0028] Based on the filtered point cloud data, use the non-uniform B-spline surface fitting algorithm to generate the three-dimensional model of the pier, align the reconstructed model with the coordinate system of the design drawing, and use the ICP algorithm to optimize the registration accuracy;

[0029] Calculate the projection deviation in the normal direction of each point on the surface and map it to the HSV color space;

[0030] Output an interactive size deviation heat map.

[0031] Further, the dynamic programming path includes an initialization stage and a feedback stage;

[0032] Initialization stage:

[0033] Load the three-dimensional point cloud model of the target structure and divide the detection grid;

[0034] Pre-set the defect feature library and establish the mapping relationship between the multi-sensor data and the confidence value;

[0035] Online planning stage:

[0036] Real-time fuse the surface texture analysis results of the visual camera and the laser ranging data to update the defect confidence map;

[0037] Calculate the regional stability score according to the variance of the current ranging data and dynamically adjust the path weight;

[0038] Generate a flight path based on the improved heuristic search algorithm;

[0039] After the rescan is completed, reduce the task weight of the corresponding node and guide the drone to turn to a new target;

[0040] Generate a Pareto front solution set for conflicting objectives;

[0041] Feedback phase:

[0042] Continuously collect sensor data during flight. If a sudden obstacle and a confidence mutation are detected, trigger path replanning;

[0043] Perform point cloud integrity verification on the scanned area that has been completed, and automatically rescan the missing area.

[0044] Furthermore, the pier measurement method that fuses the laser gyro measuring rod and the UAV vision also includes a wind speed compensation mechanism, and the wind speed compensation mechanism includes:

[0045] Automatically calculate the initial wind speed threshold according to the aerodynamic performance of the UAV, and form a dynamically adjustable wind speed threshold range;

[0046] When the wind speed is continuously monitored to exceed the current dynamic wind speed threshold:

[0047] Start a duration cumulative counter;

[0048] If the cumulative value reaches the "continuous risk" level predefined in the fuzzy rule base, trigger a response action;

[0049] Smoothly switch between the [low-speed maintenance mode] and the [emergency avoidance mode] according to the output of the fuzzy controller;

[0050] Introduce a feedforward compensation mechanism to offset the pose deviation caused by continuous wind pressure;

[0051] Enable the kinematic chain reverse compensation algorithm to eliminate the coupling effect of the aircraft body shaking on the detection data.

[0052] A pier measurement system that fuses a laser gyro measuring rod and UAV vision, the system includes:

[0053] Measurement module. In the measurement module, the UAV is equipped with a laser gyro measuring rod integrated with a gyroscope, an inclination sensor, and a wireless communication module, and flies to the pier measurement area to collect the three-dimensional attitude data and inertial reference data of the UAV in real time through the laser gyro measuring rod;

[0054] Scanning and calculation module. In the scanning and calculation module, the UAV synchronously starts the binocular vision module and the laser rangefinder, and performs multi-angle scanning on the pier along the preset circumferential path to obtain the surface feature point cloud and real-time distance data of the pier;

[0055] Space-time alignment module, which aligns the inertial reference data, visual point cloud, and real-time distance data in space and time;

[0056] A heat map generation module, which adopts an anti-interference extended Kalman filtering algorithm, fuses multi-source data, separates the actual size data of the bridge pier from the UAV motion noise, and generates the three-dimensional actual size data of the bridge pier and the heat map of the size deviation;

[0057] A rescan module, which dynamically plans the local refined scanning path of the UAV according to the confidence level of the heat map of the size deviation, and avoids obstacles in real time through a laser rangefinder until the measurement accuracy meets the standard.

[0058] The technical effects and advantages of the bridge pier measurement method and system that fuse a laser gyro measurement rod and UAV vision provided by the present invention:

[0059] Through the spatio-temporal fusion of laser gyro-vision point clouds, GAN-enhanced anti-rain and fog verification, and dynamic anti-wind closed-loop control, the present invention realizes a coordinated breakthrough in sub-millimeter-level accuracy of bridge detection, integrity of all-weather data, and all-weather continuous operation ability, solves the problem of serious distortion of measurement data caused by perception degradation under complex meteorological conditions, and thus can accurately extract the true geometric features of the bridge pier. Through the multi-dimensional collaborative optimization of anti-attack ability, dynamic security mechanism, and verification efficiency, the present invention solves the core defects of traditional video watermarking technology, such as easy erasure due to insufficient reversibility and confusion of forged attribution caused by static keys, and provides a reliable technical guarantee for digital content copyright protection. Based on the spatio-temporal alignment and fusion of a laser gyro inertial reference and vision point clouds, the present invention realizes the efficient separation of the true size of the bridge pier and motion noise, and still maintains sub-millimeter-level measurement accuracy under strong wind disturbances, which is significantly better than traditional optical measurement methods; combined with adversarial image enhancement and multi-sensor redundancy verification, it effectively overcomes the problem of feature loss in rain and fog environments and ensures data integrity under complex meteorological conditions; constructs a closed-loop detection optimization link, and through the synergistic effect of dynamic path planning and intelligent anti-wind strategies, greatly improves the detection efficiency and operation continuity under sudden environmental disturbances, forming an all-weather and all-terrain bridge pier detection ability. Description of the Drawings

[0060] Figure 1 It is a flow chart of the bridge pier measurement method that fuses a laser gyro measurement rod and UAV vision in Embodiment 1;

[0061] Figure 2 It is a flow chart of the method for generating the three-dimensional actual size data of the bridge pier and the heat map of the size deviation in Embodiment 1;

[0062] Figure 3 It is a schematic connection diagram of the bridge pier measurement system that fuses a laser gyro measurement rod and UAV vision in Embodiment 2. Detailed Embodiments

[0063] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0064] Embodiment 1:

[0065] Please refer to Figure 1 As shown, the method for measuring bridge piers by fusing a laser gyro measurement rod and UAV vision in this embodiment includes:

[0066] The UAV is equipped with a laser gyro measurement rod integrated with a gyroscope, an inclination sensor, and a wireless communication module, and flies to the bridge pier measurement area, and the three-dimensional attitude data and inertial reference data of the UAV are collected in real time through the laser gyro measurement rod;

[0067] The UAV synchronously starts the binocular vision module and the laser rangefinder, scans the bridge pier from multiple perspectives along a preset circumferential path, and obtains the surface feature point cloud and real-time distance data of the bridge pier;

[0068] Align the inertial reference data, visual point cloud, and real-time distance data in space and time;

[0069] The anti-interference extended Kalman filter algorithm is used to fuse multi-source data and separate the actual size data of the bridge pier from the UAV motion noise, and generate the three-dimensional actual size data of the bridge pier and the thermal map of the size deviation;

[0070] According to the confidence level of the thermal map of the size deviation, dynamically plan the local refined scanning path of the UAV, and avoid obstacles in real time through the laser rangefinder until the measurement accuracy reaches the standard (compared with the design drawing).

[0071] The binocular vision module calls an image enhancement model based on a generative adversarial network in a rain and fog environment to de-fog and texture repair low visibility images.

[0072] When the binocular vision module detects that the environmental humidity or particulate matter concentration exceeds the preset threshold, the image enhancement process is automatically triggered:

[0073] First, perform gray level equalization processing on the original image to suppress the uneven brightness caused by rain and fog; subsequently, input the preprocessed image into the pre-trained generative adversarial network model. The generative adversarial network model learns the mapping relationship between rain and fog degradation features and real textures through adversarial training of a large number of synthetic rain and fog image and clear image pairs, realizes image de-fogging and detail repair, and the output image after processing significantly improves the edge sharpness and texture coherence.

[0074] The repaired image and the laser rangefinder data enter the feature matching stage synchronously. By dynamically adjusting the weights of the stereo matching algorithm for binocular vision, high-frequency feature regions such as the pier structure ridge lines and bolt holes in the enhanced image are preferentially selected as the matching reference points to reduce noise interference. At the same time, the feature point cloud of the enhanced image is spatio-temporally aligned with the inertial reference data of the laser gyro measuring rod, and the extended Kalman filter algorithm is used to compensate for the delay error introduced by image processing.

[0075] The spatio-temporal alignment method includes:

[0076] Compensate for the pose jitter of the UAV itself based on the high-frequency attitude data of the gyroscope;

[0077] Dynamically bind the camera coordinate system of the binocular vision module, the laser ranging coordinate system, and the inertial reference coordinate system of the laser gyro measuring rod;

[0078] During the hovering stage of the UAV, use the static data of the laser gyro measuring rod to perform online zero-bias calibration on the binocular vision module and the laser rangefinder.

[0079] The UAV pose estimation algorithm uses the PnP algorithm to estimate the pose. The gyroscope outputs the real-time attitude angles (pitch, roll, yaw) and angular velocities of the UAV at a fixed frequency, thereby constructing a pose time series. The data frames of the binocular camera and the laser rangefinder carry accurate timestamps. For each sampling point in the visual point cloud and the laser ranging data, the pose at the corresponding moment is interpolated from the pose time series of the gyroscope according to its timestamp.

[0080] If the UAV jitters during the acquisition time of a certain visual point cloud frame, resulting in distortion of the visual point cloud, then associate the observation time of each pixel in the point cloud (in the row scanning sequence) with the high-frequency attitude data (the pose at a certain moment in the pose time series), and then complete the alignment in time through the reverse compensation displacement based on the kinematic model.

[0081] The method of dynamic binding includes:

[0082] Based on the inertial data of the laser gyro measuring rod, forcibly correct the pose estimation result of visual SLAM.

[0083] The indicators of visual SLAM include the number of feature point tracks, the reprojection error, and the trace of the pose estimation covariance matrix. If any of the indicators exceeds the limit, it is determined that visual SLAM fails and forced correction is triggered. Before determining that visual SLAM fails (only when there are abnormalities), when the pose estimation of visual SLAM is abnormal due to motion blur or feature loss, automatically switch to the inertial-dominated mode according to the motion state detection of the inertial data (such as the mutation amplitude of angular velocity or acceleration). At this time, the pose output of visual SLAM is suspended, and the high-frequency attitude data of the gyroscope is directly injected into the pose solution module to ensure the continuity of the coordinate system.

[0084] Dynamically align the origin of the camera coordinate system with the inertial reference coordinate system through Lie group transformation, and at the same time introduce the distance observation value of the laser rangefinder as a spatial constraint. This process suppresses the cumulative error of pure inertial data and avoids positioning drift caused by short-term switching.

[0085] In the inertial-dominated mode, continuously monitor the stability of the visual point cloud. When the motion state returns to being gentle, visual SLAM gradually re-participates in pose solution, but accepts real-time offset compensation of the inertial reference coordinate system to achieve smooth transition. This mechanism ensures that the spatio-temporal alignment error of multi-source data during high-speed maneuvers remains within the engineering acceptable range.

[0086] Exemplary:

[0087] In a scanning task at the top of a pier, the drone executes an emergency obstacle avoidance maneuver due to a sudden side wind.

[0088] At this time:

[0089] The visual module detects that the point cloud matching fails for 5 consecutive frames (the angular velocity exceeds the example value of 28 / s, and the acceleration exceeds 1.8g). The system immediately switches to the inertial-dominated mode, and uses the 400Hz data of the gyroscope to forcibly correct the pose. The laser ranging observation value constrains the mapping of the inertial coordinate system to suppress the horizontal drift. After about 4 control cycles (example value 60ms), the point cloud matching success rate of the visual module rises back to 85%. The system resumes the visual-inertial fusion mode, and finally the spatial offset error between the point cloud and the inertial data is controlled within the example threshold of ±7cm.

[0090] When visual SLAM fails, based on the pose of the gyroscope (the pose that has been time-aligned), forcibly correct the output of visual SLAM through extended Kalman filtering. When forcibly correcting, the observation value switches to the inertial pose, and the observation noise covariance is increased to reflect the drift characteristics of the inertial data. At the same time, filtering update is performed. The filtering update equation includes:

[0091] ;

[0092] In the formula, is the posterior state estimate, that is, the optimal state estimate after fusing the observation values at time ; is the prior state estimate, that is, the state estimate obtained only through prediction at time (without considering the current observation); is the Kalman gain; is the observation vector; is the observation matrix.

[0093] After forced correction, when visual SLAM recovers (the number of feature tracks rebounds), use the inertial pose during forced correction as the initial guess for visual relocalization; quickly match the feature points of the current frame and the historical map through the PnP algorithm to complete pose alignment.

[0094] Dynamic binding can maintain centimeter-level positioning accuracy even when vision fails, avoid UAV out-of-control or measurement interruption, and at the same time allow the UAV to perform high-speed maneuvering tasks (such as rapid circumferential scanning), shorten the measurement time, ensure strict alignment of the point cloud model with the design drawing coordinate system, and support automated defect detection.

[0095] Such as Figure 2 As shown, the generation of the 3D actual size data and the size deviation heat map of the pier includes:

[0096] The UAV flies along the axial direction of the pier at a constant distance and scans the surface at a fixed frequency;

[0097] Align the IMU data with the laser point cloud timestamp, and use the visual image to assist in point cloud coloring;

[0098] Initialize the state covariance matrix of the filter and set the prior distribution of the motion noise; take the actual size parameters of the pier (length, width, curvature) and the UAV motion noise (yaw angle jitter and acceleration offset) as the joint state variables;

[0099] Preprocess the scanned data, introduce an adaptive noise covariance matrix, and dynamically downweight the abnormal jump points (caused by surface reflection or temporary occlusion);

[0100] Iteratively execute the prediction-update steps, where in the observation update stage, higher weights are given to the visual-aided registration results, decouple the static size data of the pier and the dynamic motion noise of the UAV through the observation model, and output the denoised 3D actual size data of the pier;

[0101] Based on the filtered point cloud data, use the non-uniform B-spline surface fitting algorithm to generate the 3D model of the pier, align the reconstructed model with the design drawing coordinate system, and use the ICP algorithm to optimize the registration accuracy;

[0102] Calculate the projection deviation in the normal direction of each point on the surface and map it to the HSV color space;

[0103] Output an interactive 3D heat map (size deviation heat map).

[0104] At the same time, when generating the size deviation heat map, dynamically adjust the grid resolution of the heat map according to the UAV flight height and laser ranging accuracy, and overlay the design drawing contour lines in the out-of-tolerance area for visual comparison.

[0105] Exemplary:

[0106] During the flight of the drone, it encounters side winds, resulting in violent fluctuations in its attitude. The algorithm corrects it in real time through IMU data to eliminate the influence of motion noise on the point cloud;

[0107] The reconstructed model shows that there is an inward depression of up to 8 mm at the bottom of the bridge pier, and the red area in the thermal image is consistent with the position of the concrete spalling confirmed by manual inspection;

[0108] The output report automatically calculates the proportion of the area of the non-compliance deviation area, providing a quantitative basis for maintenance decision-making.

[0109] The dynamic programming path includes an initialization stage, an online planning stage, and a feedback stage;

[0110] Initialization stage:

[0111] Load the three-dimensional point cloud model of the target structure and divide the detection grid;

[0112] Pre-set the defect feature library and establish the mapping relationship between multi-sensor data and confidence values;

[0113] Online planning stage:

[0114] Fuse the surface texture analysis results of the visual camera and the laser ranging data in real time to update the defect confidence map;

[0115] Calculate the regional stability score according to the variance of the current ranging data and dynamically adjust the path weight;

[0116] Generate a flight path based on an improved heuristic search algorithm;

[0117] The improved heuristic search algorithm is based on the principle of the traditional A* algorithm, integrates multiple indicators such as defect confidence, ranging stability, and flight attitude constraints into the cost function, and designs a dynamic adjustment strategy. The dynamic adjustment strategy includes:

[0118] When a high-confidence defect is detected, increase the task weight and trigger a focused scan;

[0119] In narrow areas or when ranging is unstable, increase the safety weight and prioritize obstacle avoidance;

[0120] After the repeated scan is completed, reduce the task weight of the corresponding node to guide the drone to turn to a new target;

[0121] Generate a Pareto front solution set for conflicting targets (such as the shortest path vs the highest-confidence scan) for the decision-making system to select;

[0122] Only plan the path within a limited future time, and re-plan periodically in combination with real-time data to adapt to the dynamic environment.

[0123] Feedback stage:

[0124] Continuously collect sensor data during flight. If a sudden obstacle and a confidence mutation are detected, trigger path replanning;

[0125] Perform point cloud integrity verification on the scanned area that has been completed, and automatically rescan the missing areas.

[0126] Exemplary:

[0127] Take the detection of bridge piers as an example:

[0128] When the drone is flying horizontally at a set distance from the stone pier, if the laser ranging suddenly shows abnormal fluctuations, the system determines it as interference from debris and automatically generates a detour path; after identifying high-confidence irregular features (compared with the design drawing) on the surface of the stone pier, the drone switches to a spiral descent mode and obtains multi-angle detailed images within the safe range of the pitch angle; after completing a single scan, automatically compare the point cloud coverage rate and initiate a supplementary scan for the shaded and occluded areas.

[0129] Through the multi-constraint fusion cost function and the dynamic weight mechanism, upgrade the traditional path search from "geometric space optimization" to "task-safety-efficiency collaborative optimization", and solve the planning problems of target conflicts and environmental time-variation in complex inspection scenarios.

[0130] The method for measuring bridge piers by fusing a laser gyro measurement rod and drone vision also includes a wind speed compensation mechanism, and the wind speed compensation mechanism includes:

[0131] The drone is equipped with a wind speed sensor to continuously monitor the environmental wind speed. When the wind speed exceeds the preset wind speed threshold:

[0132] Reduce the flight speed of the drone and limit the maneuver amplitude;

[0133] Enhance the weight of the laser gyro data in the extended Kalman filter algorithm to suppress the visual point cloud jitter error caused by the wind speed.

[0134] Automatically calculate the initial wind speed threshold according to the aerodynamic performance of the drone (such as the maximum wind resistance level and the stability of the pan-tilt head), and form a dynamically adjustable wind speed threshold range;

[0135] When it is continuously monitored that the wind speed exceeds the current dynamic wind speed threshold:

[0136] Start a duration cumulative counter, and the growth rate of the counter is positively correlated with the wind speed exceeding amplitude (the value greater than the current dynamic wind speed threshold);

[0137] If the cumulative value reaches the "continuous risk" level predefined in the fuzzy rule base, trigger a response action.

[0138] Smoothly switch between [low-speed maintenance mode]-[emergency avoidance mode] according to the output of the fuzzy controller;

[0139] Introduce a feedforward compensation mechanism to offset the pose deviation caused by continuous wind pressure.

[0140] The feedforward compensation mechanism includes wind pressure modeling and feedforward command generation;

[0141] Wind pressure modeling:

[0142] Obtain the three-dimensional wind speed vector in real time through an on-board anemometer ;

[0143] Combined with the aerodynamic parameters of the UAV (such as the frontal area and drag coefficient), calculate the moment generated by the wind pressure on the airframe. The method for calculating the moment includes:

[0144] ;

[0145] In the formula, is the air density, is the drag coefficient, is the frontal area, is the position vector of the wind pressure center relative to the center of gravity.

[0146] Feedforward command generation:

[0147] According to the predicted wind pressure moment , inversely solve the required motor thrust compensation amount, and superimpose the compensation amount on the output of the conventional PID controller to form the final motor control command.

[0148] Automatically increase the weight of inertial navigation data fusion to suppress the periodic jitter noise in visual / laser data;

[0149] Enable the inverse compensation algorithm of the kinematic chain to eliminate the coupling effect of airframe shaking on the detection data.

[0150] The operation method of the inverse compensation algorithm of the kinematic chain includes:

[0151] Establish a motion transfer model from the airframe to the sensor (forward kinematics);

[0152] According to the airframe motion data measured by the IMU, inversely calculate the theoretical motion that the sensor should generate;

[0153] Subtract the theoretical motion component from the original sensor data to obtain the true geometric information of the static target (bridge pier).

[0154] Exemplary:

[0155] In a certain canyon bridge detection task:

[0156] The UAV encounters intermittent strong winds (wind speed fluctuations of 6 - 15 m / s) formed by the canyon effect, and the system dynamically calculates the optimal response threshold range to be 7.2 - 9.1 m / s;

[0157] When continuous excessive wind conditions are detected, automatically switch to the canyon crossing mode:

[0158] Reduce the flight speed to the lowest value within the safety envelope and maintain the detection sensor aligned with the key areas of the bridge pier;

[0159] After motion compensation of the point cloud data, the measurement error of the bridge deck joint is still controlled within ±1.5 mm;

[0160] Automatically generate a wind field-response correlation report after the task is completed, providing data support for the optimization of subsequent task parameters.

[0161] Through the technical path of fuzzy parameter processing - dynamic event judgment - multi-modal response, it gets rid of the dependence on fixed values and realizes the autonomous and robust control of inspection operations in complex wind field environments.

[0162] Embodiment 2:

[0163] As Figure 3 shown, based on the same inventive concept as the bridge pier measurement method that fuses the laser gyro measurement rod and UAV vision in the foregoing embodiment, this application provides a bridge pier measurement system that fuses the laser gyro measurement rod and UAV vision. The system in the embodiment of this application and the method embodiment are based on the same inventive concept. Among them, the system includes:

[0164] A measurement module. In the measurement module, the UAV carries a laser gyro measurement rod integrated with a gyroscope, an inclination sensor, and a wireless communication module, flies to the bridge pier measurement area, and collects the three-dimensional attitude data and inertial reference data of the UAV in real time through the laser gyro measurement rod;

[0165] A scanning and calculation module. In the scanning and calculation module, the UAV synchronously starts the binocular vision module and the laser rangefinder, scans the bridge pier from multiple perspectives along a preset circumferential path, and obtains the surface feature point cloud and real-time distance data of the bridge pier;

[0166] A spatio-temporal alignment module. The spatio-temporal alignment module aligns the inertial reference data, the visual point cloud, and the real-time distance data in space and time;

[0167] A heat map generation module. The heat map generation module uses an anti-interference extended Kalman filter algorithm to fuse multi-source data, separate the actual size data of the bridge pier from the UAV motion noise, and generate the three-dimensional actual size data of the bridge pier and the heat map of the size deviation;

[0168] A rescan module. The rescan module dynamically plans the local refined scanning path of the UAV according to the confidence level of the heat map of the size deviation, and avoids obstacles in real time through the laser rangefinder until the measurement accuracy reaches the standard.

[0169] Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these modifications and variations.

[0170] The above are only the preferred specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application, according to the technical solution and its concept of the present application, makes equivalent substitutions or changes, and should be covered by the protection scope of the present application.

Claims

1. A method for measuring bridge piers by fusing a laser gyro measurement rod and UAV vision, characterized in that, Including: A laser gyro measuring rod carried by a drone, which integrates a gyroscope, an inclination sensor, and a wireless communication module. The drone flies to the pier measurement area and collects the three-dimensional attitude data and inertial reference data of the drone in real time through the laser gyro measuring rod; The drone synchronously activates the binocular vision module and the laser rangefinder, scans the pier from multiple perspectives along a preset surrounding path, and obtains the surface feature point cloud and real-time distance data of the pier; Perform spatio-temporal alignment on the inertial reference data, visual point cloud, and real-time distance data; Adopt an anti-interference extended Kalman filter algorithm to fuse multi-source data, separate the actual size data of the pier from the motion noise of the drone, and generate the three-dimensional actual size data and size deviation heat map of the pier; The method for generating the three-dimensional actual size data and size deviation heat map of the pier includes: The drone flies at a constant distance along the axis of the pier and scans the surface at a fixed frequency; Align the IMU data with the time stamp of the laser point cloud, and the visual image is used to assist in coloring the point cloud; Initialize the state covariance matrix of the filter and set the prior distribution of the motion noise; regard the actual size parameters of the pier and the motion noise of the drone as a joint state quantity; Preprocess the scanned data, introduce an adaptive noise covariance matrix, and dynamically reduce the weight of abnormal jump points; Iteratively execute the prediction-update step, decouple the static size data of the pier from the dynamic motion noise of the drone through the observation model, and output the denoised three-dimensional actual size data of the pier; Based on the filtered point cloud data, use the non-uniform B-spline surface fitting algorithm to generate the three-dimensional model of the pier, align the reconstructed model with the coordinate system of the design drawing, and use the ICP algorithm to optimize the registration accuracy; Calculate the projection deviation in the normal direction of each point on the surface and map it to the HSV color space; Output an interactive size deviation heat map; According to the confidence level of the size deviation heat map, dynamically plan the local refined scanning path of the drone, and avoid obstacles in real time through the laser rangefinder until the measurement accuracy reaches the standard.

2. The pier measurement method for the laser gyro measurement rod and UAV vision fusion according to claim 1, characterized in that, The binocular vision module calls an image enhancement model based on a generative adversarial network in rainy and foggy environments to remove fog and repair textures for low visibility images.

3. The pier measurement method by fusing the laser gyro measurement rod and the UAV vision according to claim 1, wherein, The spatio-temporal alignment method includes: Compensate for the pose jitter of the drone itself based on the high-frequency attitude data of the gyroscope; Dynamically bind the camera coordinate system of the binocular vision module, the laser ranging coordinate system, and the inertial reference coordinate system of the laser gyro measuring rod; During the hovering stage of the drone, use the static data of the laser gyro measuring rod to perform online zero-bias calibration on the binocular vision module and the laser rangefinder.

4. The pier measurement method by fusing the laser gyro measurement rod and the UAV vision according to claim 3, characterized in that The method for dynamic binding includes: Based on the inertial data of the laser gyro measuring rod, forcibly correct the pose estimation result of visual SLAM; The indicators of visual SLAM include the number of feature point tracking, reprojection error, and the trace of the pose estimation covariance matrix. If any indicator exceeds the limit, it is determined that visual SLAM fails and forced correction is triggered; When visual SLAM fails, based on the pose of the gyroscope, forcibly correct the output of visual SLAM through the extended Kalman filter. When performing forced correction, the observation value switches to the inertial pose, and the observation noise covariance is increased to reflect the drift characteristics of the inertial data. At the same time, filtering update is performed. The filtering update equation includes: ； Wherein, is the posterior state estimate; is the prior state estimate; is the Kalman gain; is the observation vector; is the observation matrix.

5. The pier measurement method by fusing the laser gyro measurement rod and the UAV vision according to claim 1, wherein, The dynamic programming path includes an initialization stage and a feedback stage; Initialization stage: Load the three-dimensional point cloud model of the target structure and divide the detection grid; Pre-set the defect feature library and establish the mapping relationship between multi-sensor data and confidence values; Online planning stage: Real-time fuse the surface texture analysis results of the vision camera and the laser ranging data to update the defect confidence map; Calculate the regional stability score according to the variance of the current ranging data and dynamically adjust the path weight; Generate a flight path based on an improved heuristic search algorithm; After the re-scanning is completed, reduce the task weight of the corresponding node and guide the UAV to turn to a new target; Generate a Pareto front solution set for conflicting targets; Feedback stage: Continuously collect sensor data during the flight. If sudden obstacles and confidence mutations are detected, trigger path re-planning; Perform point cloud integrity verification on the scanned area that has been completed and automatically rescan the missing area.

6. The pier measurement method by fusing a laser gyroscope measuring rod with UAV vision according to claim 1, characterized in that It also includes a wind speed compensation mechanism, and the wind speed compensation mechanism includes: Automatically calculate the initial wind speed threshold according to the aerodynamic performance of the UAV to form a dynamically adjustable wind speed threshold range; When the wind speed is continuously monitored to exceed the current dynamic wind speed threshold: Start the duration cumulative counter; If the cumulative value reaches the "continuous risk" level predefined in the fuzzy rule base, trigger a response action; Smoothly switch between the [low-speed maintenance mode] and the [emergency avoidance mode] according to the output of the fuzzy controller; Introduce a feed-forward compensation mechanism to offset the pose deviation caused by continuous wind pressure; Enable the kinematic chain reverse compensation algorithm to eliminate the coupling effect of the airframe shake on the detection data.

7. A pier measurement system integrating a laser gyro measurement rod and UAV vision, characterized in that, The system includes: Measurement module. In the measurement module, the UAV is equipped with a laser gyro measurement rod integrated with a gyroscope, an inclination sensor and a wireless communication module. It flies to the bridge pier measurement area and collects the three-dimensional attitude data and inertial reference data of the UAV in real time through the laser gyro measurement rod; Scanning and calculation module. In the scanning and calculation module, the UAV synchronously starts the binocular vision module and the laser rangefinder, scans the bridge pier from multiple perspectives along the preset surrounding path, and obtains the surface feature point cloud and real-time distance data of the bridge pier; Space-time alignment module. The space-time alignment module aligns the inertial reference data, the visual point cloud and the real-time distance data in space and time; Thermodynamic map generation module. The thermodynamic map generation module uses an anti-interference extended Kalman filter algorithm to fuse multi-source data and separate the actual size data of the bridge pier from the UAV motion noise, and generate the three-dimensional actual size data of the bridge pier and the thermodynamic map of the size deviation; The method for generating the three-dimensional actual size data and the thermodynamic map of the size deviation of the bridge pier includes: The UAV flies at a constant distance along the axial direction of the bridge pier and scans the surface at a fixed frequency; Align the IMU data with the laser point cloud timestamp, and the visual image is used to assist in coloring the point cloud; Initialize the state covariance matrix of the filter and set the prior distribution of the motion noise; regard the actual size parameters of the bridge pier and the UAV motion noise as a joint state quantity; Preprocess the scanned data, introduce an adaptive noise covariance matrix, and dynamically downweight the abnormal jump points; Iteratively execute the prediction-update step, decouple the static size data of the bridge pier from the dynamic motion noise of the UAV through the observation model, and output the denoised three-dimensional actual size data of the bridge pier; Based on the filtered point cloud data, a non-uniform B-spline surface fitting algorithm is used to generate a 3D model of the bridge pier. The reconstructed model is aligned with the coordinate system of the design drawing, and the ICP algorithm is used to optimize the registration accuracy; Calculate the projection deviation in the normal direction of each point on the surface and map it to the HSV color space; Output an interactive heat map of dimensional deviation; A rescan module, which dynamically plans the local refined scanning path of the UAV according to the confidence level of the heat map of dimensional deviation and avoids obstacles in real time through a laser rangefinder until the measurement accuracy meets the standard.

Images