Multi-sensor fusion road surface real-time detection method, system and device

Abstract

The invention discloses a multi-sensor fusion road surface real-time detection method, system and device, and the method comprises the steps: carrying out the continuous data collection of a road surface through a laser radar in the advancing process of a construction machine, and obtaining the polar coordinate data; converting the polar coordinate data into rectangular coordinate data, and screening to obtain coordinate points consistent with a pavement coordinate system; correcting the coordinate points according to the height change of the laser radar; according to the angle data obtained through coordinate transformation and the laser radar attitude angle measurement data, cross section gradient data are obtained through calculation; acquiring height data according to the corrected coordinate points to calculate the flatness of the cross section; calculating a moving distance according to the laser radar GPS coordinate, constructing a pavement three-dimensional coordinate in combination with the corrected coordinate point, constructing an interpolation point by using an interpolation method, calculating to obtain an interpolation height, reconstructing the pavement three-dimensional coordinate by using the interpolation height, and calculating to obtain the flatness and gradient of a vertical section. According to the invention, efficient and real-time gradient and flatness detection can be realized, the measurement precision is improved, and the function dimension is expanded.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of road surface construction detection, and particularly relates to a multi-sensor fusion real-time road surface detection method, system and device. BACKGROUND

[0002] With the development of highway construction, the quality requirements and specifications of road construction are continuously improved. Among them, the quality of road surface construction has an important influence on the service life of the road surface, driving comfort and future maintenance, so the detection of road surface slope and flatness is particularly important. At present, the commonly used road surface flatness detection methods in construction include three-meter ruler detection, level detection and continuous flatness detector detection. However, the detection process using the level and three-meter ruler mainly relies on manual operation, and the detection speed is slow, the detection efficiency is low, and the manual detection has great uncontrollability and is easily affected by external factors. Although the detection method using the continuous flatness detector has improved the detection means to a certain extent, the detection result has poor repeatability and slow detection speed. Moreover, the current detection methods are mainly used for detecting the road surface after paving, and cannot detect the road surface flatness during paving, so as to reflect the paving quality in real time.

[0003] For example, the utility model patent with publication number CN201339159Y discloses a laser road surface flatness detector which can be installed on a detection vehicle or a towing detection device to detect and evaluate road flatness indicators. Five laser displacement sensors which can emit collimated detection laser beams downward are arranged side by side on a rigid detection beam. When the detector moves forward with the vehicle, the five laser displacement sensors simultaneously detect the distance from the rigid detection beam to the road surface in the longitudinal direction. After the collected data is processed by a computer, the road surface flatness indicators can be quickly and accurately detected. The laser road surface flatness detector proposed in the prior art needs to ensure that the laser displacement sensors simultaneously detect in the longitudinal direction during the detection process. In the case that the road surface has a certain fluctuation or a certain slope, the detection result will be affected due to the change of the detection direction of the laser displacement sensors, and the obtained data cannot reflect the actual distance. Moreover, five laser displacement sensors are used for detection, which has a relatively high cost. Meanwhile, the detection is performed on the paved road section, and the road surface flatness quality cannot be reflected in real time. In addition, this method can only detect the flatness in the longitudinal cross-sectional direction, and cannot detect the longitudinal and transverse flatness and slope simultaneously. Moreover, the corresponding position data such as latitude and longitude are lacking, and the specific position cannot be accurately positioned. SUMMARY

[0004] The purpose of this invention is to address the problems in the prior art by providing a multi-sensor fusion method, system, and device for real-time road surface detection. Using lidar as the core detection component, the invention achieves efficient and real-time slope and flatness detection through multi-sensor collaborative operation, combined with deep learning recognition and dynamic data correction algorithms, thereby improving measurement accuracy and expanding functional dimensions.

[0005] To achieve the above objectives, the present invention provides the following technical solution: Firstly, a multi-sensor fusion method for real-time road surface detection is provided, including: By using lidar to continuously collect data from the road surface during the movement of construction machinery, polar coordinate data can be obtained. p , θ ); The polar coordinate data collected by the lidar ( p , θ Convert to rectangular coordinate data x , y ), and filter out the road surface point data to obtain coordinate points consistent with the road surface coordinate system; The coordinate points are corrected based on the change in the height of the lidar relative to the initial moment, resulting in corrected coordinate points. Based on the angle data obtained during coordinate transformation and the lidar attitude angle measurement data, the cross-sectional slope data at the corresponding moment is calculated; the cross-sectional flatness is calculated based on the height data obtained from the corrected coordinate points. The moving distance of the lidar is calculated based on the recorded location of the lidar. The three-dimensional coordinates of the road surface are constructed by combining the corrected coordinate points. Interpolation points are constructed using the interpolation method. The interpolation height is calculated based on the interpolation points. The three-dimensional coordinates of the road surface are reconstructed using the interpolation height. The smoothness and slope of the road surface longitudinal profile are then calculated.

[0006] As a preferred embodiment, the polar coordinate data acquired by the lidar ( p , θ Convert to rectangular coordinate data x , y Then, the Random Sample Consensus (RANSAC) algorithm is used to filter out the road surface point data, and coordinate points consistent with the road surface coordinate system are obtained. Specifically, the following steps are included: The polar coordinate data are processed according to the following formula ( p , θ Convert to rectangular coordinate data x , y ):

[0007] Approximating the road surface as a straight line model, based on rectangular coordinate data (x , y Two points are randomly selected from the data. P 1( X 1, Y 1) P 2( X 2, Y 2) Calculate the slope and intercept of the line to determine the equation of the line. ,in, k , b The calculation expression is as follows:

[0008]

[0009] After obtaining the equation of the straight line Then, iterate through all data points. P ( X , Y ), calculate the perpendicular distance from each data point to the assumed line. d Vertical distance is used as a measure of error. d The following relationship exists between the assumed line and the line:

[0010] Based on the set threshold e ,if d ≤ e If the value is positive, it is a road surface point; otherwise, it is a non-road surface point. After the line has been filtered once based on the current assumption, it is re-selected from the rectangular coordinate data ( x , y Two data points are randomly selected from the data points for iterative filtering until the preset number of filtering times or the number of road points reaches the expected proportion. After obtaining the selected road surface points, the least squares method is used to perform linear fitting on the selected road surface point data to obtain the fitted straight line. y = kx + b And obtain the slope of the fitted line. k The calculation expression is as follows:

[0011]

[0012] Obtain the fitted slope k Then, the tilt angle is obtained according to the principle of inverse trigonometric function transformation. a The calculation expression is as follows:

[0013] Construct a rotation matrix based on the tilt angle. The expression is as follows:

[0014] Based on the constructed rotation matrix For each rectangular coordinate data ( x , y The data points in the coordinate system are transformed to obtain coordinate points consistent with the road surface coordinate system. x i ’ , y i ’ The calculation expression is as follows: .

[0015] As a preferred embodiment, the step of correcting the coordinate points based on the height change of the lidar relative to the initial time to obtain the corrected coordinate points involves measuring the height change of the lidar relative to the initial time. T For the coordinate points obtained that are consistent with the road surface coordinate system ( x i ’ , y i ’ Based on the change in altitude of the lidar relative to the initial moment T Make corrections, and correct the coordinates to ( x i ’ , y 2i ’ ),in, .

[0016] As a preferred embodiment, in the step of calculating the cross-sectional slope data at the corresponding moment based on the angle data calculated during coordinate transformation and the lidar attitude angle measurement data, the angle data calculated during coordinate transformation is the tilt angle. a The lidar attitude angle measurement data is the roll angle. roll Cross-sectional slope data at the corresponding time It is calculated according to the following expression: .

[0017] As a preferred embodiment, the step of calculating the cross-sectional flatness based on the height data obtained from the corrected coordinate points involves adjusting the corrected coordinate points (…). x i ’ , y 2i ’Height data y 2i ’ Calculate the standard deviation using the following expression. Through height data y 2i ’ Standard deviation To assess the flatness of the cross-section:

[0018] In the formula, Indicates the corrected coordinate point ( x i ’ , y 2i ’ Height data y 2i ’ , Indicates the corrected coordinate point ( x i ’ , y 2i ’ Height data y 2i ’ The average value.

[0019] As a preferred option, the location of the lidar is represented by the lidar's GPS coordinates; In the step of calculating the lidar's movement distance based on the recorded lidar GPS coordinates and constructing the road surface's three-dimensional coordinates using corrected coordinate points, the movement distance between the current lidar GPS coordinates and the initial position is calculated according to the following expression. d :

[0020] Furthermore, the three-dimensional coordinates of the road surface constructed by combining the corrected coordinate points are ( d , x i ’ , y 2i ’ ).

[0021] As a preferred approach, in the step of constructing interpolation points using an interpolation method, the Akima interpolation method is adopted. In the step of constructing interpolation points uniformly distributed from the center outwards in both positive and negative directions using the Akima interpolation method, and calculating the interpolation height based on these interpolation points, the Akima interpolation method is used to construct interpolation points uniformly distributed from the center 0 outwards in both positive and negative directions. x2i ’ According to the interpolation point x 2i ’ The interpolation height is calculated. y 3i ’ ; For each interpolation point x 2i ’ Define the corresponding slope m k The weighted average slope of adjacent intervals is calculated using the following expression:

[0022] In the formula, The slope of the original adjacent points: , ; The weights reflect the influence of the distance between adjacent intervals on the slope: , ; In each interval [ x k , x k+1 Built-in difference polynomial:

[0023] The coefficients in the formula are determined by the following conditions: a k = y k , b k = m k , a k+1 = y k+1 , b k+1 = m k+1 coefficient c k and d k satisfy: Solving for the problem yields: ,

[0024] In the formula, h k = x k+1 -x k ; After obtaining the interpolation polynomial, the interpolation height corresponding to different interpolation points is calculated based on the interpolation polynomial. y 3i ’ :

[0025] Based on interpolation points x 2i ’ Corresponding interpolation height y 3i ’ Obtain the new three-dimensional coordinates of the road surface ( d , x 2i ’ , y 3i ’ ), and by the new road surface three-dimensional coordinates ( d , x 2i ’ , y 3i ’ Reconstruct the 3D point cloud of the road surface; After obtaining the new three-dimensional point cloud data of the road surface, the longitudinal profile data at the intermediate position, i.e., the interpolation point, is used. x 2i ’ The longitudinal profile data corresponding to =0, based on the corresponding interpolated height. y 3i ’ Calculate the standard deviation and use the results to evaluate the smoothness of the longitudinal profile.

[0026] As a preferred approach, a fixed distance is set as an interval. When the detected longitudinal section distance reaches the set distance, the movement distance is adjusted accordingly. d and interpolation points x 2i ’ =0 corresponds to the interpolation height y 3i ’ Linear fitting was performed using the least squares method, and the angle corresponding to the fitted slope was used as the slope data of the longitudinal section of the corresponding segment.

[0027] Secondly, a multi-sensor fusion road surface real-time detection system is provided, including: The real-time data acquisition module is used to continuously collect road surface data via lidar during the movement of construction machinery, and to obtain polar coordinate data. p ,θ ); The coordinate data conversion module is used to convert the polar coordinate data acquired by the lidar into coordinates. p , θ Convert to rectangular coordinate data x , y ), and filter out the road surface point data to obtain coordinate points consistent with the road surface coordinate system; The coordinate point correction module is used to correct the coordinate points based on the change in the height of the lidar relative to the initial moment, and obtain the corrected coordinate points. The cross-sectional slope and flatness calculation module is used to calculate the cross-sectional slope data at the corresponding moment based on the angle data obtained during coordinate transformation and the lidar attitude angle measurement data; and to calculate the cross-sectional flatness based on the height data obtained from the corrected coordinate points. The longitudinal profile slope and smoothness calculation module is used to calculate the moving distance of the lidar based on the recorded location of the lidar, construct the three-dimensional coordinates of the road surface by combining the corrected coordinate points, construct interpolation points using the interpolation method, calculate the interpolation height based on the interpolation points, reconstruct the three-dimensional coordinates of the road surface using the interpolation height, and calculate the smoothness and slope of the road longitudinal profile.

[0028] Thirdly, a multi-sensor fusion-based real-time road surface detection device is provided to implement the multi-sensor fusion-based real-time road surface detection method as described in the first aspect, including: LiDAR: Installed perpendicular to the road surface, the emitted light beam scans the width of the road surface, and scans 360° in the vertical plane to measure distances and obtain distance information between each point on the road surface and the LiDAR. GPS module and GPS antenna: acquire latitude and longitude data through real-time dynamic differential positioning, calculate the actual moving distance, and realize lidar position positioning; Tilt sensor: measures three-axis attitude angles in real time to correct tilt errors caused by LiDAR fluctuations; Laser elevation measuring instrument receiver and camera: The laser elevation measuring instrument receiver measures the vertical vibration displacement of the lidar by rotating the laser of the laser elevation measuring instrument. It uses the camera to capture the screen data of the laser elevation measuring instrument receiver, and uses the target detection network in the deep learning model to identify the display results of the seven-segment digital tube on the screen in real time. The display results are saved and recorded for the purpose of correcting the lidar data.

[0029] Compared with the prior art, the present invention has at least the following beneficial effects: First, it achieves efficient and real-time slope and smoothness detection: Utilizing 360° non-contact dynamic scanning technology with linear array lidar, continuous data collection across the entire width of the road surface is performed during the movement of construction machinery. A single scan acquires hundreds of detection points across the cross-section, simultaneously completing the measurement of slope and smoothness in both the transverse and longitudinal sections. Compared to traditional point-by-point measurement methods using a three-meter straightedge or level, the detection efficiency is improved by 5-8 times. Simultaneously, it integrates lidar scanning data, lidar height changes relative to the initial moment, lidar attitude angle measurement data, and lidar location coordinates in real time. Combined with selected road surface point data and interpolation points constructed using interpolation methods, slope and smoothness parameters are calculated and the results are fed back during paving operations, avoiding rework caused by traditional post-construction inspections and significantly improving the efficiency of construction quality control.

[0030] Secondly, it significantly improves the accuracy of slope and smoothness measurement: a multi-source error compensation mechanism is constructed to eliminate measurement interference. Based on the angle data calculated during coordinate transformation and the lidar attitude angle measurement data, the lidar scanning angle is dynamically compensated to correct the detection direction offset caused by road surface undulations and accurately calculate the cross-sectional slope. The coordinate points are then corrected based on the lidar's height change relative to the initial moment to eliminate the impact of construction machinery vibration on smoothness measurement. Combined with the recorded lidar location, a three-dimensional point cloud model of the road surface with geographic coordinates is constructed by calculating the lidar's movement distance, enabling accurate positioning and traceability of slope and smoothness measurement results, solving the problems of low accuracy and lack of location labels in traditional methods.

[0031] Third, expanding the functionality of slope and evenness detection: It achieves simultaneous multi-parameter detection and multi-dimensional analysis. A single LiDAR scan can cover the entire width of the road surface, simultaneously calculating cross-sectional slope and evenness parameters. By using interpolation methods to construct interpolation points, the interpolation height is calculated from these points, and the three-dimensional coordinates of the road surface are reconstructed using the interpolation height to calculate the longitudinal profile evenness and slope of the road surface. This overcomes the limitation of traditional continuous evenness meters that can only detect longitudinal indicators. Furthermore, while outputting slope and evenness data, it can generate a three-dimensional point cloud model with geographic coordinates, providing comprehensive data support for tracing the source of road defects and assessing construction quality.

[0032] This invention, through multi-sensor collaborative operation and dynamic error correction technology, can adapt to complex scenarios and solve the problems of poor flexibility and limited measurement range of traditional detection equipment. This invention can achieve simultaneous and accurate measurement of cross-sectional slope, smoothness, and travel distance. Through dynamic iterative acquisition and 3D point cloud reconstruction, this invention achieves multi-dimensional continuous quantitative evaluation of road surface smoothness, overcoming the limitations of traditional discrete detection and improving detection efficiency and depth. Therefore, it provides an efficient and accurate quality control solution for projects with stringent road surface quality requirements, such as highways and airport runways. Attached Figure Description

[0033] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention. For those skilled in the art, other related drawings can be obtained from these drawings without creative effort.

[0034] Figure 1 Flowchart of the multi-sensor fusion method for real-time road surface detection according to an embodiment of the present invention; Figure 2 A schematic diagram of the multi-sensor fusion road surface real-time detection device according to an embodiment of the present invention; Figure 3 A schematic diagram of the construction process of the multi-sensor fusion road surface real-time detection device according to an embodiment of the present invention.

[0035] In the attached diagram: A - Laser elevation measuring instrument receiver; B - GPS antenna; C - GPS module; D - Tilt sensor; E - Camera; F - LiDAR. Detailed Implementation

[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, those skilled in the art can obtain other embodiments without creative effort.

[0037] Please see Figure 1 This invention proposes a multi-sensor fusion method for real-time road surface detection, comprising: By using lidar to continuously collect data from the road surface during the movement of construction machinery, polar coordinate data can be obtained. p , θ ); The polar coordinate data collected by the lidar ( p , θ Convert to rectangular coordinate data x , y The Random Sampling Consensus (RANSAC) algorithm is used to filter out road surface point data, and coordinate points consistent with the road surface coordinate system are obtained. The coordinate points are corrected based on the change in the height of the lidar relative to the initial moment, resulting in corrected coordinate points. Based on the angle data obtained during coordinate transformation and the lidar attitude angle measurement data, the cross-sectional slope data at the corresponding moment is calculated; the cross-sectional flatness is calculated based on the height data obtained from the corrected coordinate points. The moving distance of the lidar is calculated based on the recorded GPS coordinates of the lidar. The three-dimensional coordinates of the road surface are constructed by combining the corrected coordinate points. The Akima interpolation method is used to construct interpolation points that are evenly distributed from the center to both positive and negative directions. The interpolation height is calculated based on the interpolation points. The three-dimensional coordinates of the road surface are reconstructed using the interpolation height, and the smoothness and slope of the road longitudinal profile are calculated.

[0038] This invention's multi-sensor fusion-based real-time road surface detection method employs a multi-sensor post-integration deployment, dynamically adjusting the detection range and accuracy to adapt to complex scenarios and solve the problems of poor flexibility and limited measurement range of traditional detection equipment. By constructing a multi-sensor collaborative acquisition mechanism, it simultaneously acquires multi-source data such as road surface, vibration, attitude, and positioning, changing the traditional single acquisition mode and providing rich data for accurate calculation. The calculation process eliminates interference such as equipment vibration, achieving simultaneous and accurate measurement of cross-sectional slope, smoothness, and movement distance. Through dynamic iterative acquisition and 3D point cloud reconstruction, it achieves multi-dimensional continuous quantitative evaluation of road surface smoothness, overcoming the limitations of traditional discrete detection and improving detection efficiency and depth.

[0039] In one possible implementation, embodiments of the present invention utilize polar coordinate data acquired by a lidar ( p , θ Convert to rectangular coordinate data x , y The process involves using the Random Sample Consensus (RANSAC) algorithm to filter out road surface point data and obtain coordinate points consistent with the road surface coordinate system, including the following steps: The polar coordinate data are processed according to the following formula ( p , θ Convert to rectangular coordinate data x , y ):

[0040] The transformation result for the same cross-section contains hundreds of scanning points, resulting in hundreds of rectangular coordinate data. Through random sampling and model validation, the optimal model is fitted from the rectangular coordinate data containing noise and outliers, and road surface point data is selected. Since the road surface is relatively flat, the road surface is approximated as a straight line model. After completing the coordinate transformation to obtain rectangular coordinate data, the rectangular coordinate data ( x , y Two points are randomly selected from the data. P 1( X 1, Y 1) P 2( X 2, Y 2) Calculate the slope and intercept of the line to determine the equation of the line. ,in, k ,b The calculation expression is as follows:

[0041]

[0042] After obtaining the equation of the straight line Then, iterate through all data points. P ( X , Y ), calculate the perpendicular distance from each data point to the assumed line. d Vertical distance is used as a measure of error. d The following relationship exists between the assumed line and the line:

[0043] Based on the set threshold e ,if d ≤ e If the value is positive, it is a road surface point; otherwise, it is a non-road surface point. After the line has been filtered once based on the current assumption, it is re-selected from the rectangular coordinate data ( x , y Two data points are randomly selected from the data points for iterative filtering until the preset number of filtering times or the number of road points reaches the expected proportion. After obtaining the selected road surface points, the least squares method is used to perform linear fitting on the selected road surface point data to obtain the fitted straight line. y = kx + b And obtain the slope of the fitted line. k The calculation expression is as follows:

[0044]

[0045] Obtain the fitted slope k Then, the tilt angle is obtained according to the principle of inverse trigonometric function transformation. a The calculation expression is as follows:

[0046] Construct a rotation matrix based on the tilt angle. The expression is as follows:

[0047] Based on the constructed rotation matrix For each rectangular coordinate data ( x , y The data points in the coordinate system are transformed to obtain coordinate points consistent with the road surface coordinate system. xi ’ , y i ’ The calculation expression is as follows: .

[0048] In one possible implementation, the present invention involves correcting the coordinate points based on the height change of the lidar relative to the initial time, obtaining the corrected coordinate points, and measuring the height change of the lidar relative to the initial time. T For the coordinate points obtained that are consistent with the road surface coordinate system ( x i ’ , y i ’ Based on the change in altitude of the lidar relative to the initial moment T Make corrections, and correct the coordinates to ( x i ’ , y 2i ’ ),in, .

[0049] In one possible implementation, the corrected rectangular coordinates (x, y, y) can be obtained in a single cross-sectional scan. i ',y 2i '), tilt angle a GPS data ( lon , lat ), attitude angle data ( pitch , roll , yaw The present invention includes a step of calculating the cross-sectional slope data at a corresponding moment based on the angle data calculated during coordinate transformation and the lidar attitude angle measurement data. The angle data calculated during coordinate transformation is the tilt angle. a The lidar attitude angle measurement data is the roll angle. roll Cross-sectional slope data at the corresponding time It is calculated according to the following expression: .

[0050] In one possible implementation, the step of calculating the cross-sectional flatness based on height data obtained from the corrected coordinate points in this embodiment of the invention involves adjusting the corrected coordinate points (…). x i ’ , y 2i ’ Height datay 2i ’ Calculate the standard deviation using the following expression. Through height data y 2i ’ Standard deviation To assess the flatness of the cross-section:

[0051] In the formula, Indicates the corrected coordinate point ( x i ’ , y 2i ’ Height data y 2i ’ , Indicates the corrected coordinate point ( x i ’ , y 2i ’ Height data y 2i ’ The average value.

[0052] Each cycle of the above steps yields scan data of a road surface cross-section profile. Therefore, by continuously repeating these steps and using the GPS coordinates recorded in each cycle, the distance traveled between the current LiDAR GPS coordinates and the initial position is calculated using the following expression. d :

[0053] Based on the distance traveled d Corrected rectangular coordinates ( x i ’ , y 2i ’ Constructing the three-dimensional coordinates of the road surface ( d , x i ’ , y 2i ’ Because the rotation scanning angle is inconsistent during lidar scanning, different cross-sections do not have the same... x i ’ It is difficult to calculate the standard deviation and slope after that, so, after obtaining the three-dimensional coordinates of the road surface ( d ,x i ’ , y 2i ’ Then, using the Akima interpolation method, interpolation points are constructed that are uniformly distributed from the center 0 in both positive and negative directions. x 2i ’ According to the interpolation point x 2i ’ The interpolation height is calculated. y 3i ’ .

[0054] In one possible implementation, the Akima interpolation method is used to construct interpolation points uniformly distributed from the center in both positive and negative directions. In the step of calculating the interpolation height based on these interpolation points, the Akima interpolation method is used to construct the interpolation points uniformly distributed from the center 0 in both positive and negative directions. x 2i ’ According to the interpolation point x 2i ’ The interpolation height is calculated. y 3i ’ ; For each interpolation point x 2i ’ Define the corresponding slope m k The weighted average slope of adjacent intervals is calculated using the following expression:

[0055] In the formula, The slope of the original adjacent points: , ; The weights reflect the influence of the distance between adjacent intervals on the slope: , ; In each interval [ x k , x k+1 Built-in difference polynomial:

[0056] The coefficients in the formula are determined by the following conditions: a k = y k, b k = m k , a k+1 = y k+1 , b k+1 = m k+1 coefficient c k and d k satisfy: Solving for the problem yields: ,

[0057] In the formula, h k = x k+1 - x k ; After obtaining the interpolation polynomial, the interpolation height corresponding to different interpolation points is calculated based on the interpolation polynomial. y 3i ’ :

[0058] Based on interpolation points x 2i ’ Corresponding interpolation height y 3i ’ Obtain the new three-dimensional coordinates of the road surface ( d , x 2i ’ , y 3i ’ ), and by the new road surface three-dimensional coordinates ( d , x 2i ’ , y 3i ’ Reconstruct the 3D point cloud of the road surface; After obtaining the new three-dimensional point cloud data of the road surface, the longitudinal profile data at the intermediate position, i.e., the interpolation point, is used. x 2i ’ The longitudinal profile data corresponding to =0, based on the corresponding interpolated height. y 3i ’Calculate the standard deviation and use the results to evaluate the smoothness of the longitudinal profile.

[0059] For calculating the longitudinal profile slope, a fixed distance (e.g., 5m) is set as an interval. When the detected longitudinal profile distance reaches the set distance, the slope is adjusted according to the moving distance. d and interpolation points x 2i ’ =0 corresponds to the interpolation height y 3i ’ Linear fitting was performed using the least squares method, and the angle corresponding to the fitted slope was used as the slope data of the longitudinal section of the corresponding segment.

[0060] According to the embodiments of the present invention, the flatness and slope of the cross section of the road surface at different locations can be calculated through the first to fourth steps of the embodiments of the present invention. After the iterative processing of the first to fourth steps, a three-dimensional point cloud of the road surface can be constructed. After processing the point cloud data at different locations, the fifth step of calculating the longitudinal slope of the road surface can be achieved.

[0061] Another embodiment of the present invention also proposes a multi-sensor fusion real-time road surface detection system, comprising: The real-time data acquisition module is used to continuously collect road surface data via lidar during the movement of construction machinery, and to obtain polar coordinate data. p , θ ); The coordinate data conversion module is used to convert the polar coordinate data acquired by the lidar into coordinates. p , θ Convert to rectangular coordinate data x , y The Random Sampling Consensus (RANSAC) algorithm is used to filter out road surface point data, and coordinate points consistent with the road surface coordinate system are obtained. The coordinate point correction module is used to correct the coordinate points based on the change in the height of the lidar relative to the initial moment, and obtain the corrected coordinate points. The cross-sectional slope and flatness calculation module is used to calculate the cross-sectional slope data at the corresponding moment based on the angle data obtained during coordinate transformation and the lidar attitude angle measurement data; and to calculate the cross-sectional flatness based on the height data obtained from the corrected coordinate points. The longitudinal profile slope and smoothness calculation module is used to calculate the moving distance of the lidar based on the recorded lidar GPS coordinates, construct the three-dimensional coordinates of the road surface by combining the corrected coordinate points, and construct interpolation points evenly distributed from the center to both positive and negative directions using the Akima interpolation method. The interpolation height is calculated based on the interpolation points, and the three-dimensional coordinates of the road surface are reconstructed using the interpolation height to calculate the smoothness and slope of the road longitudinal profile.

[0062] Please see Figure 2 and Figure 3 This invention also proposes a multi-sensor fusion-based real-time road surface detection device for implementing the multi-sensor fusion-based real-time road surface detection method, comprising: LiDAR F: Installed perpendicular to the road surface, the emitted light beam scans the width of the road surface, and scans 360° in the vertical plane to measure distances and obtain distance information between each point on the road surface and the LiDAR. GPS module C and GPS antenna B: acquire latitude and longitude data through real-time dynamic RTK differential technology, and calculate the actual moving distance by combining the Vincenty algorithm and the Vincenty function to achieve lidar position positioning; Tilt sensor D: measures three-axis attitude angles in real time to correct tilt errors caused by LiDAR fluctuations; Laser elevation measuring instrument receiver A and camera E: The laser elevation measuring instrument receiver measures the vertical vibration displacement of the lidar by rotating the laser of the laser elevation measuring instrument. It uses the target detection network in the YOLOv12 deep learning model to identify the display results of the seven-segment digital tube on the screen in real time using the laser elevation measuring instrument receiver screen data captured by the camera, and saves and records the display results for correcting the lidar data.

[0063] It should be noted that, Figure 1 The main structure shown is merely for assembling related components. The structure for mounting it on construction machinery requires specific design based on the different types of machinery and their requirements. However, the dimensions of the mounting structure must not interfere with the normal scanning operation of the lidar and must allow for the proper installation of related sensors. When mounting on construction machinery, the entire device must be securely fixed. Vibration damping devices (such as rubber shock-absorbing pads) can be used to reduce the impact of machinery vibration on measurements. Simultaneously, the connection lines of each component must be organized and secured to prevent damage or poor contact due to shaking during operation.

[0064] In one possible implementation, the multi-sensor fusion road surface real-time detection device deployment scheme of this invention adopts a rear-integrated installation strategy, orderly installing devices such as LiDAR F, laser elevation meter receiver A, GPS antenna B, GPS module C, tilt sensor D, and camera E behind construction or inspection machinery. This approach can adapt to various detection scenarios. Specifically, the laser elevation meter receiver A requires strict initial position calibration to ensure its initial scanning point is accurately aligned with the zero point of the displacement receiver. Simultaneously, the scanning angle of LiDAR F is dynamically set according to the actual width of the road surface to be detected, achieving an optimal balance between detection range and accuracy.

[0065] In one possible implementation, the data acquisition process initiates a multi-sensor collaborative working mode on the measured road section. The lidar F acquires road surface data at high frequency through cross-sectional scanning; the laser elevation meter receiver A monitors the device's height change relative to the initial moment in real time, accurately capturing vibration information during equipment operation; the tilt sensor D precisely measures the overall device's attitude angle changes, providing attitude reference for subsequent data calibration; the GPS module C uses differential GPS technology to correct the acquired GNSS data in real time, significantly improving positioning accuracy; and the camera E simultaneously acquires image information from the laser elevation meter receiver A's display screen, providing visual evidence for data verification.

[0066] In one possible implementation, the slope and smoothness calculation model constructs a cross-sectional slope calculation model based on coordinate transformation angle data and attitude angle data to accurately solve the cross-sectional slope parameters at the current location; based on the corrected height data, a professional algorithm is used to calculate the road surface smoothness index; and by combining attitude angle data and calibrated GPS data, the movement distance of the detection device is calculated through trajectory extrapolation methods to achieve simultaneous measurement of multiple parameters.

[0067] In one possible implementation, the 3D point cloud reconstruction and longitudinal section analysis method employs a dynamic iterative acquisition mechanism. During mechanical movement, the aforementioned data acquisition, processing, and calculation steps are continuously repeated to obtain cross-sectional height, slope, smoothness, and movement distance data at different locations. Based on this data, a 3D point cloud model of the road surface is reconstructed, and the point cloud data of the same cross-section are interpolated to generate continuous longitudinal section height data. A specialized algorithm is used to calculate the smoothness index and standard deviation within the longitudinal section, achieving a multi-dimensional quantitative assessment of road surface smoothness.

[0068] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0069] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for real-time road surface detection using multi-sensor fusion, characterized in that, include: By using lidar to continuously collect data from the road surface during the movement of construction machinery, polar coordinate data can be obtained. ρ , θ ); The polar coordinate data collected by the lidar ( ρ , θ Convert to rectangular coordinate data x , y ), and filter out the road surface point data to obtain coordinate points consistent with the road surface coordinate system; The coordinate points are corrected based on the change in the height of the lidar relative to the initial moment, resulting in corrected coordinate points. Based on the angle data obtained during coordinate transformation and the lidar attitude angle measurement data, the cross-sectional slope data at the corresponding moment is calculated; the cross-sectional flatness is calculated based on the height data obtained from the corrected coordinate points. The moving distance of the lidar is calculated based on the recorded location of the lidar. The three-dimensional coordinates of the road surface are constructed by combining the corrected coordinate points. Interpolation points are constructed using the interpolation method. The interpolation height is calculated based on the interpolation points. The three-dimensional coordinates of the road surface are reconstructed using the interpolation height. The smoothness and slope of the road surface longitudinal profile are then calculated.

2. The multi-sensor fusion method for real-time road surface detection according to claim 1, characterized in that, The polar coordinate data collected by the lidar ( ρ , θ Convert to rectangular coordinate data x , y Then, the Random Sample Consensus (RANSAC) algorithm is used to filter out the road surface point data, and coordinate points consistent with the road surface coordinate system are obtained. Specifically, the following steps are included: The polar coordinate data are processed according to the following formula ( ρ , θ Convert to rectangular coordinate data x , y ): Approximating the road surface as a straight line model, based on rectangular coordinate data ( x , y Two points are randomly selected from the data. P 1( X 1, Y 1) P 2( X 2, Y 2) Calculate the slope and intercept of the line to determine the equation of the line. ,in, k , b The calculation expression is as follows: After obtaining the equation of the straight line Then, iterate through all data points. P ( X , Y ), calculate the perpendicular distance from each data point to the assumed line. d Vertical distance is used as a measure of error. d The following relationship exists between the line and the assumed line: Based on the set threshold ε ,if d ≤ ε If the value is positive, it is a road surface point; otherwise, it is a non-road surface point. After the line has been filtered once based on the current assumption, it is re-selected from the rectangular coordinate data ( x , y Two data points are randomly selected from the data points for iterative filtering until the preset number of filtering times or the number of road points reaches the expected proportion. After obtaining the selected road surface points, the least squares method is used to perform linear fitting on the selected road surface point data to obtain the fitted straight line. y = kx + b And obtain the slope of the fitted line. k The calculation expression is as follows: Obtain the fitted slope k Then, the tilt angle is obtained according to the principle of inverse trigonometric function transformation. a The calculation expression is as follows: Construct a rotation matrix based on the tilt angle. The expression is as follows: Based on the constructed rotation matrix For each rectangular coordinate data ( x , y The data points in the coordinate system are transformed to obtain coordinate points consistent with the road surface coordinate system. x i ’ , y i ’ The calculation expression is as follows: 。 3. The multi-sensor fusion method for real-time road surface detection according to claim 2, characterized in that, The step of correcting the coordinate points based on the height change of the lidar relative to the initial time to obtain the corrected coordinate points involves measuring the height change of the lidar relative to the initial time. T For the coordinate points obtained that are consistent with the road surface coordinate system ( x i ’ , y i ’ Based on the change in altitude of the lidar relative to the initial moment T Make corrections, and correct the coordinates to ( x i ’ , y 2i ’ ),in, .

4. The multi-sensor fusion method for real-time road surface detection according to claim 3, characterized in that, The step of calculating the cross-sectional slope data at the corresponding moment based on the angle data calculated during coordinate transformation and the lidar attitude angle measurement data, wherein the angle data calculated during coordinate transformation is the tilt angle. a The lidar attitude angle measurement data is the roll angle. roll Cross-sectional slope data at the corresponding time It is calculated according to the following expression: 。 5. The multi-sensor fusion method for real-time road surface detection according to claim 3, characterized in that, The step of calculating the cross-sectional flatness based on the height data obtained from the corrected coordinate points, for the corrected coordinate points ( x i ’ , y 2i ’ Height data y 2i ’ Calculate the standard deviation using the following expression. Through height data y 2i ’ Standard deviation To assess the flatness of the cross-section: In the formula, Indicates the corrected coordinate point ( x i ’ , y 2i ’ Height data y 2i ’ , Indicates the corrected coordinate point ( x i ’ , y 2i ’ Height data y 2i ’ The average value.

6. The multi-sensor fusion method for real-time road surface detection according to claim 3, characterized in that, The location of the lidar is represented by the lidar's GPS coordinates; In the step of calculating the lidar's movement distance based on the recorded lidar GPS coordinates and constructing the road surface's three-dimensional coordinates using corrected coordinate points, the movement distance between the current lidar GPS coordinates and the initial position is calculated according to the following expression. d : Furthermore, the three-dimensional coordinates of the road surface constructed by combining the corrected coordinate points are ( d , x i ’ , y 2i ’ ).

7. The multi-sensor fusion method for real-time road surface detection according to claim 6, characterized in that, In the step of constructing interpolation points using an interpolation method, the Akima interpolation method is adopted. In the step of constructing interpolation points uniformly distributed from the center outwards in both positive and negative directions using the Akima interpolation method, and calculating the interpolation height based on these interpolation points, the Akima interpolation method is used to construct interpolation points uniformly distributed from the center 0 outwards in both positive and negative directions. x 2i ’ According to the interpolation point x 2i ’ The interpolation height is calculated. y 3i ’ ; For each interpolation point x 2i ’ Define the corresponding slope m k The weighted average slope of adjacent intervals is calculated using the following expression: In the formula, The slope of the original adjacent points: , ; The weights reflect the influence of the distance between adjacent intervals on the slope: , ; In each interval [ x k , x k+1 Built-in difference polynomial: The coefficients in the formula are determined by the following conditions: a k = y k ， b k = m k ， a k+1 = y k+1 ， b k+1 = m k+1 coefficient c k and d k satisfy: Solving for the problem yields: ， In the formula, h k = x k+1 - x k ; After obtaining the interpolation polynomial, the interpolation height corresponding to different interpolation points is calculated based on the interpolation polynomial. y 3i ’ : Based on interpolation points x 2i ’ Corresponding interpolation height y 3i ’ Obtain the new three-dimensional coordinates of the road surface ( d , x 2i ’ , y 3i ’ ), and by the new road surface three-dimensional coordinates ( d , x 2i ’ , y 3i ’ Reconstruct the 3D point cloud of the road surface; After obtaining the new three-dimensional point cloud data of the road surface, the longitudinal profile data at the intermediate position, i.e., the interpolation point, is used. x 2i ’ The longitudinal profile data corresponding to =0, based on the corresponding interpolated height. y 3i ’ Calculate the standard deviation and use the results to evaluate the smoothness of the longitudinal profile.

8. The multi-sensor fusion method for real-time road surface detection according to claim 7, characterized in that, A fixed distance is set as an interval. When the detected longitudinal section distance reaches the set distance, the movement distance is adjusted accordingly. d and interpolation points x 2i ’ =0 corresponds to the interpolation height y 3i ’ Linear fitting was performed using the least squares method, and the angle corresponding to the fitted slope was used as the slope data of the longitudinal section of the corresponding segment.

9. A multi-sensor fusion real-time road surface detection system, characterized in that, include: The real-time data acquisition module is used to continuously collect road surface data via lidar during the movement of construction machinery, and to obtain polar coordinate data. ρ , θ ); The coordinate data conversion module is used to convert the polar coordinate data acquired by the lidar into coordinates. ρ , θ Convert to rectangular coordinate data x , y ), and filter out the road surface point data to obtain coordinate points consistent with the road surface coordinate system; The coordinate point correction module is used to correct the coordinate points based on the change in the height of the lidar relative to the initial moment, and obtain the corrected coordinate points. The cross-sectional slope and flatness calculation module is used to calculate the cross-sectional slope data at the corresponding moment based on the angle data obtained during coordinate transformation and the lidar attitude angle measurement data; and to calculate the cross-sectional flatness based on the height data obtained from the corrected coordinate points. The longitudinal profile slope and smoothness calculation module is used to calculate the moving distance of the lidar based on the recorded location of the lidar, construct the three-dimensional coordinates of the road surface by combining the corrected coordinate points, construct interpolation points using the interpolation method, calculate the interpolation height based on the interpolation points, reconstruct the three-dimensional coordinates of the road surface using the interpolation height, and calculate the smoothness and slope of the road longitudinal profile.

10. A multi-sensor fusion-based real-time road surface detection device, used to implement the multi-sensor fusion-based real-time road surface detection method as described in any one of claims 1 to 8, comprising: LiDAR: Installed perpendicular to the road surface, the emitted light beam scans the width of the road surface, and scans 360° in the vertical plane to measure distances and obtain distance information between each point on the road surface and the LiDAR. GPS module and GPS antenna: acquire latitude and longitude data through real-time dynamic differential positioning, calculate the actual moving distance, and realize lidar position positioning; Tilt sensor: measures three-axis attitude angles in real time to correct tilt errors caused by LiDAR fluctuations; Laser elevation measuring instrument receiver and camera: The laser elevation measuring instrument receiver measures the vertical vibration displacement of the lidar by rotating the laser of the laser elevation measuring instrument. It uses the camera to capture the screen data of the laser elevation measuring instrument receiver, and uses the target detection network in the deep learning model to identify the display results of the seven-segment digital tube on the screen in real time. The display results are saved and recorded for the purpose of correcting the lidar data.

Images