A high-speed road surface pit inspection device and method thereof

Abstract

The present application relates to the technical field of road pit detection, and particularly relates to a high-speed road pit inspection device and a method thereof, which comprises an inspection robot straddling on the top of a guardrail and reciprocally moving along the guardrail, a plurality of types of sensors and an edge computing module are carried on the robot, and are used for synchronously collecting and real-time pre-processing of road multi-source sensing data; the inspection device further comprises a digital twin platform in communication connection with the robot, which is internally provided with a geometric model construction module, a historical database module, a data fusion module, a model training module, a pit formation prediction model module and a maintenance fund demand deduction module and the like; the digital twin platform outputs pit risk prediction results of each road section according to a prediction model obtained by fusing real-time sensing data and historical data, dynamically generates a differential inspection task plan according to the pit risk prediction results, deduces fund demands under different maintenance strategies and outputs multi-scenario analysis results. The present application realizes a change from passive post-detection to active pre-prediction, and can automatically perform quantitative deduction of maintenance funds.

Description

Technical Field

[0001] This invention relates to the field of road surface pothole detection technology, and in particular to a device and method for inspecting potholes on highways. Background Technology

[0002] Currently, the inspection of potholes on highways mainly relies on manual foot patrols, regular inspections by dedicated inspection vehicles, or drone aerial photography. Among these methods, manual patrols are inefficient and pose significant personal safety risks; dedicated inspection vehicles are expensive, with low inspection frequency and high costs; and drone inspections are significantly limited by battery life and adverse weather conditions. All of these methods are reactive detection models, meaning that potholes are only discovered and addressed after they have formed and pose a threat to driving safety.

[0003] Furthermore, current maintenance management budgeting relies primarily on managers' experience and estimations, lacking scientific data support and quantitative analysis methods. This results in insufficient precision in maintenance fund allocation, easily leading to wasted funds or untimely maintenance of critical road sections. Simultaneously, there is a lack of effective data closure between existing detection methods and maintenance decisions; detection data is not fully utilized to guide the optimization of maintenance plans and the rational allocation of funds.

[0004] In recent years, digital twin technology has been explored for application in the field of infrastructure operation and maintenance. However, a systematic technical solution has not yet been formed to deeply integrate it with high-frequency and low-cost road data collection methods and further extend it to the intelligent prediction of maintenance funding needs. Summary of the Invention

[0005] In view of this, the purpose of this invention is to propose a highway pavement pothole inspection device and method to solve the problem that current inspection methods are all post-inspection modes and cannot guide the generation of maintenance plans.

[0006] To achieve the above objectives, the present invention provides a highway road pothole inspection device, comprising: The inspection robot sits atop the highway guardrail and moves back and forth along it. The inspection robot is equipped with multiple types of sensors and an edge computing module. The multiple types of sensors are used to synchronously collect multi-source perception data of the road surface during the movement, and the edge computing module is used to perform real-time preprocessing of the multi-source perception data. The digital twin platform is connected to the inspection robot via a wireless communication network and receives the preprocessed multi-source sensing data. The digital twin platform has built-in geometric model construction module, historical database module, data fusion module, model training module, pothole formation prediction model and maintenance funding demand projection module. The geometric model construction module is used to construct a three-dimensional geometric model of the highway section based on the road section design drawings and high-precision maps; the historical database module is used to integrate and store historical pavement technical condition detection data, historical pothole damage records, historical traffic flow data, historical meteorological data, and pavement structure design parameters; the data fusion module is used to align and fuse the multi-source sensing data with the historical data on a unified spatial coordinate and time axis to form a standard training dataset; the model training module is used to train the pothole formation prediction model based on the standard training dataset; the pothole formation prediction model is used to predict the pothole formation risk probability and prediction formation time window of each target road section within a set time window in the future based on the multi-source sensing data and the historical data, and output the pothole risk prediction result; The maintenance funding demand projection module is used to automatically identify the maintenance demand type and corresponding maintenance workload based on the pothole risk prediction results, and, in conjunction with the dynamically updated maintenance unit price database, project the maintenance funding demand curve under different maintenance strategies, and output the maintenance funding demand projection results including multi-scenario comparative analysis. The digital twin platform is also used to dynamically generate the inspection task plan of the inspection robot based on the pothole risk prediction results, and to allocate different inspection frequencies to road sections with different risk levels.

[0007] The various types of sensors include: A road surface image acquisition unit, which is a high-resolution industrial camera equipped with a global shutter, is used to continuously acquire visible light images of the road surface during the movement of the inspection robot. The visible light images of the road surface are used to identify existing road potholes and road surface cracks. The lidar unit, which is a multi-line lidar, is used to continuously collect three-dimensional point cloud data of the road surface during the movement of the inspection robot. The three-dimensional point cloud data of the road surface is used to generate a road surface elevation model and measure the three-dimensional dimensions of potholes. An inertial measurement unit is used to collect real-time motion attitude data of the inspection robot during its movement. The motion attitude data includes three-axis acceleration, three-axis angular velocity and heading angle, which is used to correct sensor displacement errors caused by uneven road surface and vehicle traffic. A global positioning system module is used to collect the real-time location information of the inspection robot at each data collection moment, and the real-time location information is used to associate the multi-source sensing data with geospatial location; A vibration acceleration sensor is used to collect vibration signals generated when the inspection robot moves along the guardrail. The vibration signals are used to indirectly assess the smoothness of the road surface. The multi-source sensing data includes the visible light image of the road surface, the three-dimensional point cloud data of the road surface, the motion posture data, the real-time location information, and the vibration signal.

[0008] The various types of sensors undergo unified spatiotemporal calibration: In terms of spatial calibration, a sensor spatial coordinate system is established with the centroid of the inspection robot as the origin. An improved iterative nearest point algorithm is used to achieve pixel-level spatial registration between the three-dimensional point cloud data of the road surface and the visible light image of the road surface, so that the data collected by each sensor has a unified spatial reference. In terms of time calibration, the clock of the global positioning system module is used as the reference clock, and Kalman filtering is used to achieve time synchronization of data from each sensor, with a time synchronization accuracy better than 10 milliseconds; The visible light image of the road surface, the three-dimensional point cloud data of the road surface, the motion posture data, the real-time location information, and the vibration signal, after spatiotemporal calibration, are fused on a unified spatial coordinate and time axis to form a multi-source perception dataset of the road surface with complete spatiotemporal labels.

[0009] The real-time preprocessing of the multi-source sensing data by the edge computing module includes: The visible light image of the road surface is subjected to noise reduction and contrast enhancement processing; The road surface 3D point cloud data is filtered and registered to remove outliers; The vibration signal is subjected to bandpass filtering and time-domain feature extraction to extract vibration feature parameters including vibration peak value, vibration root mean square value and vibration dominant frequency; The amount of preprocessed multi-source sensing data is less than 30% of the original collected data. The preprocessed multi-source sensing data is transmitted to the digital twin platform through the wireless communication network.

[0010] The pit formation prediction model is a hybrid prediction model that integrates physical mechanisms and data-driven approaches, including: The physical mechanism sub-model establishes a benchmark attenuation curve for pavement performance based on the pavement structure mechanics attenuation theory. The benchmark attenuation curve takes the pavement structure layer thickness, material elastic modulus and base bearing capacity as input parameters and outputs the theoretical attenuation trajectory of the pavement condition index under the condition of no external intervention. The data-driven sub-model is constructed using a long short-term memory network. It uses the standard training dataset as training samples, road segment basic information, traffic load characteristics, and climate conditions as input features, and the deviation between the actual road surface condition exponential decay value and the benchmark decay curve as the prediction target. It learns the nonlinear deviation pattern of actual decay relative to theoretical decay. The pothole formation prediction model superimposes and corrects the baseline decay curve output by the physical mechanism sub-model and the deviation prediction value output by the data-driven sub-model, and outputs the road surface condition index prediction value in the future time series. When the predicted value of the road surface condition index is lower than a preset first threshold and the weather forecast within the prediction time window includes continuous rainfall or freeze-thaw cycle conditions, the pothole formation prediction model outputs a pothole formation risk warning for the corresponding road segment. The pothole formation risk warning includes a risk level and a predicted formation time window.

[0011] The preset first threshold is a road surface condition index of less than 75, and the risk level is divided into three levels: high risk, medium risk, and low risk, based on the probability of pothole formation. When the probability of pothole formation is greater than or equal to 60%, it is marked as a high-risk level; When the probability of pothole formation is greater than or equal to 30% and less than 60%, it is marked as a medium-risk level; When the probability of pothole formation is less than 30%, it is marked as a low-risk level.

[0012] The inspection task plan assigns different inspection frequencies to road sections with different risk levels, specifically configured as follows: The road sections marked as high-risk are assigned an inspection frequency of once a day; The road sections marked as medium risk are assigned an inspection frequency of once every three days; Road sections marked as low-risk are assigned an inspection frequency of once a week.

[0013] The maintenance funding requirement simulation module includes: The maintenance demand identification submodule is used to identify a list of road sections that require maintenance intervention in each month, quarter, and unit year based on the pothole formation risk level and predicted formation time window of each road section in the pothole risk prediction results, and to match the corresponding maintenance measure type for each road section in the list. The maintenance measure types include: preventive maintenance measures for road sections where the predicted pavement condition index will drop to the range of 75 to 80 and potholes have not yet formed; repair maintenance measures for road sections where potholes are predicted to form; and structural overhaul measures for road sections where the predicted pavement condition index will drop to below 60. The engineering quantity estimation submodule is used to estimate the implementation engineering quantity of each maintenance measure based on the pavement structure parameters and defect prediction information of each road segment in the road segment list, combined with the preset unit engineering quantity index. The implementation engineering quantity includes the pothole repair area, crack sealing length and preventive maintenance coverage area. The dynamic unit price database module is used to store the unit prices of materials, labor, and machinery shifts for various maintenance measures, and dynamically updates the unit prices of materials, labor, and machinery shifts according to a preset cycle. The funding projection submodule is used to project funding requirements under different maintenance strategies based on the implemented project quantity and the current unit price data in the dynamic unit price database module, using a multi-scenario analysis method.

[0014] The multi-scenario analysis method executed by the capital projection submodule includes: Three maintenance strategy scenarios are set up: aggressive prevention strategy, conventional repair strategy, and conservative strategy. The aggressive prevention strategy is to implement preventive maintenance measures immediately before the predicted formation time window for all road sections marked as high-risk and medium-risk. The conventional repair strategy is as follows: only repair and maintenance measures are implemented on road sections marked as high-risk, while road sections of medium risk are not intervened and are monitored. The conservative strategy is to not implement preventive intervention on any road sections marked as high-risk or medium-risk, and to wait for potholes to actually form before implementing repair maintenance. The funding projection submodule calculates the maintenance funding needs within a future set time period under each maintenance strategy scenario, and generates monthly, quarterly, and annual funding demand distribution data for each scenario. The funding projection submodule also performs a life-cycle cost comparison analysis on each maintenance strategy scenario, calculates the cumulative maintenance cost under each scenario within the preset life-cycle period, and outputs the maintenance strategy scenario with the lowest cumulative maintenance cost as the recommended maintenance plan.

[0015] Based on the same invention, this invention also provides a method for inspecting potholes on highways, comprising the following steps: Step S1: The digital twin platform generates a daily inspection task plan based on the latest pothole risk prediction results output by the pothole formation prediction model. In the inspection task plan, high-risk road sections, medium-risk road sections and low-risk road sections are assigned different inspection frequencies. Step S2: The inspection robot moves along the highway guardrail according to the inspection task plan. During the movement, the multi-type sensors synchronously collect multi-source perception data of the road surface. The multi-source perception data includes at least visible light images of the road surface, three-dimensional point cloud data of the road surface, motion posture data, real-time position information and vibration signals. Step S3: The edge computing module performs real-time preprocessing on the multi-source sensing data. The preprocessing includes image denoising and enhancement, point cloud filtering and registration, and vibration signal feature extraction. The preprocessed data is uploaded to the digital twin platform through a wireless communication network. Step S4: The data fusion module of the digital twin platform receives the preprocessed multi-source sensing data, aligns and fuses the multi-source sensing data with the historical data in the historical database module on a unified spatial coordinate and time axis, and updates the standard training dataset; Step S5: The model training module retrains or corrects the parameters of the pothole formation prediction model based on the updated standard training dataset, and the updated pothole formation prediction model outputs the updated pothole risk prediction result. Step S6: The maintenance funding demand projection module automatically identifies the maintenance demand type and corresponding maintenance work volume of each target road section based on the updated pothole risk prediction results. Combining the current unit price data in the dynamic unit price database module, it uses a multi-scenario analysis method to project the maintenance funding demand curve under different maintenance strategies and outputs the maintenance funding demand projection results including multi-scenario comparative analysis. Step S7: The digital twin platform automatically generates a multi-dimensional maintenance budget report based on the maintenance funding demand projection results. The multi-dimensional maintenance budget report includes the distribution of funding demand by time dimension, the details of funding demand by road segment dimension, the proportion of funding demand by maintenance measures dimension, and the comparison data of funding demand for different maintenance strategies. It also outputs the maintenance strategy with the lowest cumulative maintenance cost as the recommended maintenance plan. Step S8: Obtain actual maintenance data after the maintenance operation is carried out. The actual maintenance data includes actual maintenance workload, actual maintenance cost, pavement performance test value after maintenance, and maintenance operation quality evaluation. Input the actual maintenance data into the feedback correction module of the digital twin platform. Step S9: The feedback correction module compares the actual maintenance data with the predicted data in the maintenance funding demand projection results, and corrects and updates the model parameters of the pothole formation prediction model according to the deviation comparison results, thus completing the complete closed loop of data collection, pothole prediction, maintenance funding projection and feedback correction.

[0016] As the inspection robot moves along the highway guardrail, its multiple onboard sensors simultaneously collect multi-dimensional data about the road surface. The acquired image, point cloud, position, and vibration data are first processed in real time by the robot's onboard edge computing module. Filtering, compression, and feature extraction are performed at the data source, effectively reducing the total amount of data that needs to be transmitted. The processed data is then transmitted wirelessly to a digital twin platform. The platform's data fusion module integrates and aligns this newly acquired data with existing historical data (including annual road condition monitoring records, traffic load statistics, and meteorological observation data) according to unified spatial coordinates and time series, thereby constructing a standardized model training sample set. Based on this, the model training module uses this sample set to train a pothole formation prediction model. This model can then calculate the probability and approximate time of pothole formation on various road sections within a specific future period based on real-time road condition information and environmental conditions, outputting a spatial distribution prediction result including risk levels. The maintenance funding demand projection module uses this prediction result to automatically determine the road sections requiring maintenance measures and the applicable maintenance schemes, and calculates the required funding scale under different strategies. Meanwhile, the digital twin platform formulates differentiated inspection plans based on risk prediction results, and arranges more frequent monitoring for road sections with higher risk levels, thereby establishing a complete data closed loop from on-site data collection, information fusion and processing, disease trend prediction to inspection scheduling.

[0017] This invention achieves a leap from manual reliance to automated and intelligent road surface pothole detection through deep integration of inspection robots and a digital twin platform. The robot autonomously moves along guardrails as its track, continuously collecting road condition information. The entire inspection process does not require disrupting traffic flow or using expensive dedicated inspection vehicles, significantly reducing the manpower and equipment costs of daily inspections. The digital twin platform systematically integrates real-time collected data with historically accumulated information, enabling the predictive model to detect potholes in advance, transforming the traditional approach of detecting potholes as they appear into predicting them before they appear, thus providing ample lead time for maintenance decisions. Furthermore, the platform dynamically allocates inspection resources based on the predicted risk level of each road segment, prioritizing limited manpower and equipment for sections with the highest concentration of potential potholes, thereby improving the overall targeting and resource utilization efficiency of inspection work. For long routes, the system supports multiple robots working collaboratively in segments, ensuring that high-risk areas within each segment are inspected promptly on the same day, effectively resolving the contradiction between inspection coverage and timeliness in single-robot operation mode. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a structural block diagram of the inspection device according to an embodiment of the present invention; Figure 2 This is an information flow diagram of the maintenance funding requirement projection module in an embodiment of the present invention; Figure 3 This is a flowchart of the inspection method according to an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0021] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0022] like Figure 1 As shown, a highway road pothole inspection device includes: An inspection robot is mounted on top of a highway guardrail and moves back and forth along the guardrail. The inspection robot is equipped with multiple types of sensors and an edge computing module. The multiple types of sensors are used to synchronously collect multi-source perception data of the road surface as the inspection robot moves along the guardrail. The edge computing module is used to perform real-time preprocessing of the multi-source perception data. The moving mechanism straddles the top of the guardrail. It includes a double-sided clamping and walking mechanism, with two sets of drive wheels respectively disposed on the two side walls of the guardrail. Each set of drive wheels includes at least two drive wheels arranged along the length of the device, and each drive wheel is driven by an independent servo motor. The double-sided clamping and walking mechanism also includes a tension adjustment device, which adjusts the clamping force between the two sets of drive wheels via a spring or electric push rod, enabling the device to adapt to corrugated beam guardrails or concrete guardrails of different thicknesses. The digital twin platform is connected to the inspection robot via a wireless communication network and receives the multi-source sensing data preprocessed by the edge computing module. The digital twin platform has a built-in geometric model construction module, historical database module, data fusion module, model training module, pothole formation prediction model, and maintenance funding demand projection module. The geometric model construction module is used to construct a three-dimensional geometric model of the highway section based on the road section design drawings and high-precision maps. The three-dimensional geometric model includes the geometric information and material property information of each structural layer of the road surface. The historical database module is used to integrate and store road surface technical condition inspection data, pothole damage records, traffic flow data, meteorological data, and road structure design parameters from previous years. The data fusion module is used to align and fuse the multi-source sensing data with the historical data in the historical database module on a unified spatial coordinate and time axis to form a standard training dataset. The model training module is used to train the pothole formation prediction model based on the standard training dataset. The pothole formation prediction model predicts the probability of pothole formation risk and the predicted formation time window for each target road segment within a set time window based on the real-time input multi-source sensing data and the historical data in the historical database module, and outputs the pothole risk prediction result. The maintenance funding demand projection module automatically identifies the maintenance demand type and corresponding maintenance workload of each target road segment in different time windows based on the pothole risk prediction results. Combined with the dynamically updated maintenance unit price database, it projects the maintenance funding demand curve under different maintenance strategies and outputs the maintenance funding demand projection results including multi-scenario comparative analysis. The digital twin platform also dynamically generates an inspection task plan for the inspection robot based on the pothole risk prediction results. The inspection task plan assigns different inspection frequencies to road sections with different risk levels.

[0023] In one specific implementation, the inspection robot straddles the top of the guardrail and forms a stable rolling contact with the guardrail through a dual-sided clamping walking mechanism. Two sets of drive wheels of this mechanism are respectively located on the two side walls of the guardrail. Each set of drive wheels includes at least two drive wheels arranged along the length of the device, and each drive wheel is driven by an independent servo motor to enable the inspection robot to move forward, backward, and stop on the guardrail. The dual-sided clamping walking mechanism also includes a tension adjustment device, which adjusts the clamping force between the two sets of drive wheels through a spring or electric push rod, allowing the device to adapt to corrugated beam guardrails or concrete guardrails of different thicknesses. The wireless communication network is a 4G or 5G communication network with a data transmission frequency of not less than 1Hz. The digital twin platform is deployed on a cloud server or a local server and establishes a data connection with the inspection robot via the TCP / IP protocol. The future time window is a period of 1 to 3 months. The multiple types of sensors include a road image acquisition unit, a lidar unit, an inertial measurement unit, a global positioning system module, and a vibration acceleration sensor. The edge computing module performs real-time preprocessing on multi-source sensing data, including image denoising and enhancement, point cloud filtering and registration, and vibration signal feature extraction. The amount of data after preprocessing is less than 30% of the original collected data. For long-distance highway sections (such as single highways exceeding 50 kilometers in length), multiple inspection robots can be deployed for segmented collaborative inspections. Each robot is responsible for a designated section, and each robot communicates with the digital twin platform and independently executes inspection tasks according to the risk level within its respective section. This solves the problem that a single robot inspecting at a speed of 1-2 km / h cannot complete the entire line inspection in a single day.

[0024] As the inspection robot moves along the highway guardrail, multiple types of sensors simultaneously collect multi-source perception data of the road surface. The edge computing module preprocesses the collected data in real time to reduce the data transmission bandwidth requirements. The preprocessed data is then uploaded to the digital twin platform via a wireless communication network. The data fusion module of the digital twin platform aligns and fuses the received multi-source perception data with historical data in the historical database module on a unified spatial coordinate and time axis to form a standard training dataset. The model training module trains a pothole formation prediction model based on the standard training dataset. The trained pothole formation prediction model predicts the probability of pothole formation risk and the predicted formation time window for each road segment within a set time window based on the real-time input multi-source perception data and historical data, and outputs the pothole risk prediction results. The maintenance funding demand extrapolation module automatically identifies maintenance needs based on the pothole risk prediction results and extrapolates the funding needs under different maintenance strategies. The digital twin platform also dynamically generates inspection task plans based on the pothole risk prediction results, allocating different inspection frequencies to road segments with different risk levels, thereby realizing a complete data link from data collection, data transmission, data fusion, pothole prediction to task planning.

[0025] By collaborating with inspection robots and a digital twin platform, the inspection of potholes on highways has become automated and intelligent. The inspection robots can move autonomously along guardrails and collect road surface data in real time, eliminating the need for traffic closures or dedicated inspection vehicles, significantly reducing inspection costs and manpower. The digital twin platform, by integrating multi-source sensing data with historical data to construct a standard training dataset, enables pothole formation prediction models to predict the risk and timing of pothole formation in advance, shifting from post-inspection to pre-inspection. Simultaneously, the digital twin platform dynamically generates inspection task plans based on the prediction results, allocating differentiated inspection frequencies to road sections with different risk levels, concentrating inspection resources on high-risk sections and improving resource utilization efficiency. For long-distance road sections, deploying multiple robots for segmented collaborative inspections ensures that all sections can complete the inspection of their respective high-risk sections on the same day, resolving the conflict between the inspection speed of a single robot and full-line coverage.

[0026] The various types of sensors include: A road surface image acquisition unit, which is a high-resolution industrial camera equipped with a global shutter, is used to continuously acquire visible light images of the road surface during the movement of the inspection robot. The visible light images of the road surface are used to identify existing road potholes and road surface cracks. The lidar unit, which is a multi-line lidar, is used to continuously collect three-dimensional point cloud data of the road surface during the movement of the inspection robot. The three-dimensional point cloud data of the road surface is used to generate a road surface elevation model and measure the three-dimensional dimensions of potholes. An inertial measurement unit is used to collect real-time motion attitude data of the inspection robot during its movement. The motion attitude data includes three-axis acceleration, three-axis angular velocity and heading angle, which is used to correct sensor displacement errors caused by uneven road surface and vehicle traffic. A global positioning system module is used to collect the real-time location information of the inspection robot at each data collection moment, and the real-time location information is used to associate the multi-source sensing data with geospatial location; A vibration acceleration sensor is used to collect vibration signals generated when the inspection robot moves along the guardrail. The vibration signals are used to indirectly assess the smoothness of the road surface. The multi-source sensing data includes the visible light image of the road surface, the three-dimensional point cloud data of the road surface, the motion posture data, the real-time location information, and the vibration signal.

[0027] In one specific implementation, the high-resolution industrial camera has a resolution of no less than 12 megapixels, is equipped with a global shutter to eliminate motion blur, has a field of view of no less than 60°, and its optical axis forms an angle of 15° to 30° with the road surface normal to cover the road surface area to the side of the inspection robot. For roadside guardrail installation, the camera's effective detection distance is 1m to 5m, covering the outer edge of the driving lane; for median guardrail installation, the camera's effective detection distance is 2m to 8m, covering the overtaking lane and adjacent driving lanes, with a shooting frequency of 10 frames per second. The multi-line lidar is a 16-line lidar with a detection distance of no less than 100 meters, a ranging accuracy better than ±2 cm, a scanning frequency of 10 Hz, a horizontal field of view of 360° or configured as a 90° to 120° field of view facing the road surface depending on the installation method, and a vertical field of view of ±15°. It acquires dense three-dimensional spatial coordinate point cloud data of the road surface by emitting laser beams and receiving reflected signals, and its effective scanning range can cover the same road surface area corresponding to the camera's field of view. The inertial measurement unit (IMU) includes a three-axis MEMS gyroscope and a three-axis MEMS accelerometer. The gyroscope has a zero-bias stability better than 0.5° / h, the accelerometer has a range of ±16g, and the data output frequency is 100 Hz. It is used to measure the angular velocity and linear acceleration of the inspection robot in real time. The global positioning system (GPS) module is a differential GPS module or an RTK-GPS module, with a positioning accuracy better than 0.5 meters and a data output frequency of 10 Hz. In sections where GPS signals are blocked (such as tunnel sections), the GPS module automatically switches to an auxiliary positioning mode based on a combination of guardrail odometers and IMU positioning. It measures the travel distance through the wheel encoder and combines it with the IMU heading integration to achieve relative positioning. The cumulative positioning error rate does not exceed 1% of the travel distance. The vibration acceleration sensor is a three-axis MEMS accelerometer with a range of ±16g and a data output frequency of 100 Hz. It is installed at the connection point between the inspection robot's shell and the moving mechanism. The correlation mechanism between the vibration signal and the potholes is as follows: When a vehicle tire drives over a pothole, it generates an impact excitation. This excitation energy is transmitted along the road structure layer to the guardrail in the form of an elastic wave, and then transmitted to the walking mechanism and shell of the inspection robot via the guardrail. After the vibration acceleration sensor installed at the connection position collects the vibration signal, its time domain peak characteristics and frequency domain energy distribution characteristics can be used as indirect indicators of road surface smoothness. The impact signal generated by the pothole is characterized by a significant peak in the time domain (amplitude more than twice that of normal road surface) and a wide frequency band characteristic in the frequency domain (energy distribution in the range of 20Hz to 200Hz). Based on this, the degree of abnormality of the vibration signal can be used to infer the deterioration state of the road surface smoothness.

[0028] As the inspection robot moves along the guardrail, the road surface image acquisition unit continuously acquires visible light images of the road surface at a frequency of 10 frames per second to identify apparent defects such as potholes and cracks; the lidar unit emits a laser beam at a frequency of 10 Hz to scan the road surface, receives reflected signals to generate three-dimensional point cloud data of the road surface, processes the point cloud data to generate a road surface elevation model, and accurately measures the three-dimensional dimensions of potholes, such as length, width, and depth; the inertial measurement unit acquires motion attitude data of the inspection robot, such as three-axis acceleration, three-axis angular velocity, and heading angle, at a frequency of 100 Hz; and the global positioning system module acquires data of the inspection robot at a frequency of 10 Hz. The real-time location information at each data acquisition moment is used to add a spatial location label to each frame of sensing data; the vibration acceleration sensor collects the vibration signal generated by the inspection robot moving along the guardrail at a frequency of 100 Hz, and indirectly evaluates the smoothness of the road surface by analyzing the time domain peak and frequency domain energy distribution of the vibration signal. When the vehicle drives over a pothole, the impact excitation is transmitted through the road surface-guardrail-robot path, and the sensor collects a signal with obvious time domain peak and wide frequency domain characteristics; the data collected by the above five types of sensors together constitute multi-source sensing data, providing multi-dimensional data input for subsequent data fusion and pothole prediction.

[0029] By employing five different types of sensors, the system achieves multi-dimensional and comprehensive perception of highway pavement conditions. The pavement image acquisition unit provides visual evidence of apparent defects, the lidar unit provides precise geometric data of the pavement's three-dimensional morphology, and the inertial measurement unit and GPS module ensure that each frame of perceived data has accurate spatial position and attitude information (even in tunnel sections where GPS signals are blocked, the IMU + odometer combination positioning mode can maintain reliable positioning). The vibration acceleration sensor provides dynamic response information of pavement smoothness through the temporal peak value and frequency domain energy distribution of the vibration signal. The mechanical transmission path between the vibration signal and potholes is clear (tire → pothole impact → pavement elastic wave → guardrail → robot), making the vibration signal an effective indirect indicator of pavement smoothness. The collaborative work of multiple types of sensors enables the system to simultaneously acquire pavement appearance, geometric information, position information, and dynamic response information, providing a rich data foundation for accurate pothole identification and prediction. In particular, the introduction of the vibration acceleration sensor allows the system to detect abnormal pavement conditions even before potholes form but before pavement smoothness deteriorates, further improving the prediction lead time.

[0030] The various types of sensors undergo unified spatiotemporal calibration: In terms of spatial calibration, a sensor spatial coordinate system is established with the centroid of the inspection robot as the origin. An improved iterative nearest point algorithm is used to achieve pixel-level spatial registration between the three-dimensional point cloud data of the road surface and the visible light image of the road surface, so that the data collected by each sensor has a unified spatial reference. In terms of time calibration, the clock of the global positioning system module is used as the reference clock, and Kalman filtering is used to achieve time synchronization of data from each sensor, with a time synchronization accuracy better than 10 milliseconds; The visible light image of the road surface, the three-dimensional point cloud data of the road surface, the motion posture data, the real-time location information, and the vibration signal, after spatiotemporal calibration, are fused on a unified spatial coordinate and time axis to form a multi-source perception dataset of the road surface with complete spatiotemporal labels.

[0031] As a specific implementation, the improved iterative nearest point algorithm adds an initial registration step based on road surface texture features on the basis of the standard iterative nearest point algorithm. First, feature points in the visible light image and lidar reflectance intensity image of the road surface are extracted by the scale-invariant feature transform (SIFT) algorithm and coarse matching is performed to obtain an initial transformation matrix. Then, the initial transformation matrix is ​​used as the initial value of the iterative nearest point algorithm for fine iterative registration, thereby avoiding the problem that the standard iterative nearest point algorithm is prone to getting trapped in local optima. The registration accuracy reaches the sub-centimeter level (that is, the spatial deviation between the point cloud data and the corresponding pixel after projection onto the image plane is better than 2cm). This accuracy is sufficient to meet the actual engineering needs of pothole size measurement (usually the pothole diameter is not less than 15cm and the depth is not less than 2cm). The Kalman filter time synchronization method is as follows: using the pulse-per-second (PPS) signal from the Global Positioning System (GPS) module as the reference clock signal, the time deviation between the arrival time of each sensor data frame and the reference clock is measured. The time deviation is optimally estimated and predicted using a Kalman filter. Linear interpolation is used to calculate the corresponding value at the reference clock time in each sensor data frame. For camera images, linear interpolation is performed between adjacent frames to generate the image at the reference time; for LiDAR point clouds, linear interpolation is performed between adjacent frames to generate the point cloud coordinates at the reference time; for IMU data, linear interpolation is performed between adjacent frames to generate the attitude angle at the reference time. This achieves time alignment of the sensor data with a time synchronization accuracy better than 10 milliseconds. The spatiotemporally calibrated multi-source sensing dataset is stored in a common data format (such as ROS Bag format or a custom binary format). Each data record includes a timestamp (accurate to milliseconds), spatial coordinates (longitude, latitude, and elevation), sensor type identifier, and original data content.

[0032] In terms of spatial calibration, a sensor spatial coordinate system is established with the centroid of the inspection robot as the origin. First, feature points in the visible light image of the road surface and the reflectance intensity image of the lidar are extracted using the SIFT algorithm and coarse matching is performed to obtain an initial transformation matrix. Then, this initial transformation matrix is ​​used as the initial value for the iterative nearest point algorithm to perform fine iterative registration, achieving sub-centimeter-level spatial registration between the 3D point cloud data of the road surface and the visible light image of the road surface, so that the data collected by each sensor has a unified spatial reference. In terms of time calibration, the second pulse signal of the global positioning system module is used as the reference clock signal. The time deviation between the arrival time of each sensor data frame and the reference clock is measured. The time deviation is optimally estimated and predicted by the Kalman filter. The corresponding value of each sensor data frame at the reference clock time is calculated by linear interpolation, realizing the time synchronization of the data of each sensor. All the sensing data after spatiotemporal calibration are fused on a unified spatial coordinate and time axis to form a multi-source sensing dataset of the road surface with complete spatiotemporal labels.

[0033] By performing unified spatiotemporal calibration on multiple types of sensors, the inconsistency between multi-sensor data in space and time was resolved. Spatial calibration ensures that data collected by different sensors correspond to the same spatial location on the road surface, avoiding spatial misalignment caused by different sensor installation positions. This accuracy is sufficient to cover the engineering requirements of pothole detection. Temporal calibration uses Kalman filtering combined with linear interpolation to ensure that data collected by different sensors correspond to the same moment, avoiding temporal misalignment caused by different sensor acquisition frequencies or data transmission delays. The spatiotemporally calibrated multi-source sensing dataset has complete spatial coordinates and time labels, providing a high-quality and highly consistent data foundation for subsequent data fusion and training of pothole formation prediction models, significantly improving the accuracy and reliability of the prediction models.

[0034] The real-time preprocessing of the multi-source sensing data by the edge computing module includes: The visible light image of the road surface is subjected to noise reduction and contrast enhancement processing; The road surface 3D point cloud data is filtered and registered to remove outliers; The vibration signal is subjected to bandpass filtering and time-domain feature extraction to extract vibration feature parameters including vibration peak value, vibration root mean square value and vibration dominant frequency; The amount of preprocessed multi-source sensing data is less than 30% of the original collected data. The preprocessed multi-source sensing data is transmitted to the digital twin platform through the wireless communication network.

[0035] As a specific implementation method, the denoising process for the visible light image of the road surface employs a median filtering algorithm or a Gaussian filtering algorithm, with a filter kernel size of 3×3 pixels to 5×5 pixels, to remove salt-and-pepper noise and Gaussian noise introduced during image acquisition due to ambient light and sensor noise; the contrast enhancement process employs a histogram equalization algorithm or an adaptive histogram equalization algorithm to enhance the contrast between the edges of road surface cracks and potholes and the background, thereby improving the accuracy of subsequent defect identification. The filtering process for the 3D point cloud data of the road surface employs either a statistical filtering algorithm or a radius filtering algorithm. The statistical filtering algorithm calculates the average distance between each point and other points in its neighborhood, identifying and removing points whose average distance exceeds a certain standard deviation multiple (usually 1.0 to 2.0 times) of the global average distance. The radius filtering algorithm sets a search radius (usually 5 cm to 10 cm) and a minimum number of points threshold in the neighborhood (usually 10 to 20 points), identifying and removing points with fewer points within the search radius than the minimum number of points threshold, thus eliminating noise points caused by laser scattering or environmental interference. The registration process uses an iterative nearest-point algorithm to register the current frame point cloud with the previous frame point cloud, eliminating point cloud displacement caused by the movement of the inspection robot. Before registration, voxel filtering is used to downsample the point cloud, reducing the number of points in each frame from approximately 300,000 to approximately 10,000, thereby reducing the computational load of registration. The bandpass filtering of the vibration signal employs an infinite impulse response bandpass filter or a finite impulse response bandpass filter, with a passband frequency range of 0.5 Hz to 100 Hz, to filter out high-frequency environmental noise and low-frequency trend terms. Time-domain feature extraction includes calculating the peak value, root mean square (RMS) value, and dominant frequency of the vibration signal (extracted via Fast Fourier Transform to find the frequency corresponding to the power spectrum peak). The embedded AI computing chip in the edge computing module has a computing power of no less than 10 TOPS. The preprocessed multi-source sensing data volume is less than 30% of the original acquired data volume. After JPEG compression, the image size is approximately 10% to 20% of the original size. After voxel downsampling and quantization compression, the point cloud size is approximately 20% to 30% of the original size. After feature extraction, only the feature values ​​of the vibration signal are retained (approximately 5% to 10% of the original waveform data size). The overall compression rate after combining these three factors can reach over 70%. The wireless communication network adopts a 4G or 5G cellular network, and the upload method is a streaming transmission method of simultaneous data collection and upload. When the network signal is interrupted, the preprocessed data is temporarily stored in the local memory of the inspection robot (with a capacity of not less than 128GB) and will be transmitted again after the network is restored to ensure that the data is not lost.

[0036] The edge computing module sequentially performs median filtering for noise reduction and histogram equalization for contrast enhancement on the visible light images of the road surface to improve image quality. For the 3D point cloud data of the road surface, it sequentially performs statistical filtering or radius filtering to remove outlier noise points, then performs voxel filtering downsampling (sparsening each frame of the point cloud from approximately 300,000 points to approximately 10,000 points) followed by iterative nearest-point registration to eliminate point cloud displacement. For the vibration signal, it sequentially performs infinite impulse response bandpass filtering to extract the effective frequency band signal, then performs time-domain feature extraction to calculate characteristic parameters such as vibration peak value, root mean square value, and dominant frequency. The preprocessed multi-source sensing data is compressed to less than 30% of the original data volume, and then uploaded to the digital twin platform via a 4G / 5G wireless communication network in a streaming manner. When the network signal is interrupted, the data is temporarily stored in local storage and will be transmitted again after the network is restored.

[0037] By setting up an edge computing module on the inspection robot, real-time preprocessing of multi-source sensing data significantly reduces the amount of data that needs to be transmitted to the cloud. More than 70% of the redundant data in the original collection is filtered out or compressed at the edge through image compression, point cloud downsampling, and vibration feature extraction. This reduces the bandwidth usage and transmission cost of the wireless communication network, and improves the real-time performance and reliability of data transmission. Voxel filtering downsampling compresses each frame of point cloud from 300,000 points to 10,000 points. While preserving the macroscopic geometric features of the road surface, it reduces the computational load of point cloud registration by an order of magnitude, enabling the Iterative Closest Point (ICP) algorithm registration to run in real time on the embedded platform. The local memory's offline caching mechanism ensures that data collected in areas with unstable network signals, such as highway tunnels and mountainous areas, is not lost. Data is automatically resumed after the network is restored, ensuring the integrity of the data collection. At the same time, the edge-end preprocessing (such as image denoising and enhancement, point cloud filtering and registration, and vibration signal feature extraction) provides the cloud-based digital twin platform with high-quality data that has undergone preliminary cleaning and feature enhancement, reducing the burden of cloud data processing and improving the overall data processing efficiency of the system.

[0038] As a further preferred embodiment of the present invention, a lightweight inference model is deployed within the edge computing module. This model uses model pruning and quantization techniques to compress standard neural networks, with no more than 100 million model parameters and a model size of no more than 50MB. It can run independently on an embedded AI computing chip at the edge and is used to perform real-time preliminary inference on road images, point clouds, and vibration signals collected by various types of sensors. It identifies suspected pothole areas, crack areas, and other road anomalies in the road images and generates structured data containing anomaly locations, anomaly types, and preliminary confidence labels. This lightweight inference model runs entirely on the inspection robot and can complete preliminary defect identification without relying on any external network connection, ensuring that the system can still independently complete basic inspection and defect screening functions in road sections with unstable network signals (such as mountainous areas and tunnel sections).

[0039] In road sections with 5G or fiber optic network access, as a preferred implementation, the system also includes a large cloud-deployed model (with no less than 1 billion model parameters) to receive suspected anomaly data and structured tags uploaded by the edge computing module. This model performs cross-modal deep identification and verification of suspected anomaly areas by fusing visible light image features of the road surface, 3D point cloud geometric features, and vibration signal time-frequency features, outputting the final damage type determination, damage level assessment, and accurate 3D dimension measurement results. The cloud-based large model also continuously optimizes model parameters through incremental learning based on the massive amount of structured tags uploaded from the edge and the results of manual verification. The optimized lightweight model version is periodically distributed to the edge computing module, enabling over-the-air upgrades of the edge model.

[0040] Through the aforementioned edge-cloud collaborative inference architecture, the lightweight edge model performs millisecond-level real-time initial screening, filtering more than 95% of normal road surface data on the edge side, and uploading only suspected defect data (accounting for about 5% to 10% of all collected data) to the cloud for in-depth verification; under the condition of network interruption or no cloud access, the edge model can still independently output preliminary defect identification results for inspection personnel to refer to, and the core detection function is not lost.

[0041] The pit formation prediction model is a hybrid prediction model that integrates physical mechanisms and data-driven approaches, including: The physical mechanism sub-model establishes a benchmark attenuation curve for pavement performance based on the pavement structure mechanics attenuation theory. The benchmark attenuation curve takes the pavement structure layer thickness, material elastic modulus and base bearing capacity as input parameters and outputs the theoretical attenuation trajectory of the pavement condition index under the condition of no external intervention. The data-driven sub-model is constructed using a long short-term memory network. It uses the standard training dataset as training samples, road segment basic information, traffic load characteristics, and climate conditions as input features, and the deviation between the actual road surface condition exponential decay value and the benchmark decay curve as the prediction target. It learns the nonlinear deviation pattern of actual decay relative to theoretical decay. The pothole formation prediction model superimposes and corrects the baseline decay curve output by the physical mechanism sub-model and the deviation prediction value output by the data-driven sub-model, and outputs the road surface condition index prediction value in the future time series. When the predicted value of the road surface condition index is lower than a preset first threshold and the weather forecast within the prediction time window includes continuous rainfall or freeze-thaw cycle conditions, the pothole formation prediction model outputs a pothole formation risk warning for the corresponding road segment. The pothole formation risk warning includes a risk level and a predicted formation time window.

[0042] The pavement condition index (PCI) is calculated according to the method specified in the "Highway Technical Condition Assessment Standard" (JTG 5210-2018). Specifically, the pavement damage rate DR is first calculated according to formula (1): (1) in, For the first i The cumulative area of ​​road surface damage of this type For the first i The weight of each type of road surface damage, where A is the road surface inspection area. The total number of damage types (21 types for asphalt pavement). Then calculate PCI according to formula (2): (2) in, =15.00, =0.412 (applicable to asphalt pavements). The PCI value ranges from 0 to 100, with a higher value indicating better pavement condition. According to JTG 5210-2018, PCI ≥ 90 is excellent, ≥ 80 and < 90 is good, ≥ 70 and < 80 is average, ≥ 60 and < 70 is below average, and < 60 is poor.

[0043] As a specific implementation method, the pavement structure mechanical attenuation theory establishes a fatigue life prediction model for pavement structures under repeated vehicle loads based on the theory of multi-layered elastic layered systems. It uses the thickness of each pavement structural layer, the elastic modulus and Poisson's ratio of each layer material, and the bearing capacity of the base course (such as the California Bearing Ratio (CBR) or resilient modulus) as input parameters, and the Pavement Condition Index (PCI) as the output index to calculate the theoretical attenuation trajectory of the pavement under no maintenance intervention conditions. For in-service highway sections, parameters already specified in the design phase (such as layer thickness and design elastic modulus) can be directly obtained from the design drawings. For material parameters that may age after many years of service (such as the residual elastic modulus of the surface asphalt mixture and the residual bearing capacity of the base course), indoor tests can be conducted through on-site core drilling, or non-destructive testing and inversion calculations can be performed using a falling weight deflectometer (FWD) to obtain the material parameter values ​​under the current actual conditions. The baseline degradation curve typically exhibits an S-shaped decline in the Pavement Condition Index (PCI) with increasing service time, initially gradual and then steepening. During the initial 3-5 years of operation, the PCI decline is relatively gradual (an average decrease of 1-2 points per year), accelerating during the mid-term (5-10 years) (an average decrease of 3-5 points per year), and further accelerating after 10 years until approaching the major overhaul threshold. The Long Short-Term Memory (LSTM) network is a variant of a recurrent neural network suitable for time series forecasting. Through three gating units—input gate, forget gate, and output gate—it achieves memory and forgetting control of long-term series information, effectively learning the long-term dependencies in pavement performance degradation. During model training, the standard training dataset is divided into a training set (the first 80% of the time period) and a validation set (the last 20% of the time period) in chronological order. Basic road segment information (pavement structure layer thickness, material type, service life), traffic load characteristics (average daily traffic volume, proportion of heavy-duty trucks, axle load spectrum), and climate conditions (average monthly rainfall, average monthly temperature, number of freeze-thaw cycles) are used as input features. The deviation between the actual PCI decay value and the baseline decay curve is used as the prediction target. Mean squared error (MSE) is used as the loss function. The model parameters are trained using the backpropagation algorithm and the Adam optimizer. The preset first threshold is a pavement condition index (PCI) of less than 75. This threshold is determined based on the dividing value between "Good" and "Medium" PCI ratings in the "Highway Technical Condition Assessment Standard" (JTG 5210), but can be dynamically adjusted according to the technical level, traffic load, and safety importance of different road sections (for example, for critical road sections such as extra-large bridges, tunnels, and long downhill sections, the threshold can be raised to 80 for early warning; for road sections with low traffic volume, the threshold can be lowered to 70). The continuous rainfall condition is a continuous rainfall period of no less than 3 days and a daily rainfall of no less than 10 mm; the freeze-thaw cycle condition is an alternating weather condition where the daily minimum temperature is below 0℃ and the daily maximum temperature is above 5℃ for no less than 5 consecutive days.For newly built road sections or those with insufficient historical data accumulation (less than 2 years), the training samples of the data-driven sub-model are insufficient. In this case, the benchmark decay curve output by the physical mechanism sub-model is mainly used as the prediction result. The bias prediction value of the data-driven sub-model is initially set to zero. As the inspection data continues to accumulate (usually more than 6 months of data collection is required), the bias correction function of the data-driven sub-model is gradually enabled, and the weight of the data-driven part gradually increases from 0 to the target weight as the amount of data accumulation increases.

[0044] The physical mechanism sub-model takes pavement structure layer thickness, material elastic modulus, and base course bearing capacity as input parameters. Material parameters for in-service road sections are verified through core drilling or FWD non-destructive testing. Based on the theory of multi-layered elastic systems, it calculates the theoretical attenuation trajectory of the pavement condition index (PCI) under conditions without external intervention, outputting a baseline attenuation curve. The data-driven sub-model uses a standard training dataset as training samples and takes basic road section information, traffic load characteristics, and climate conditions as input features. It uses the deviation between the actual PCI attenuation value and the baseline attenuation curve as the prediction target, learning the nonlinear deviation of actual attenuation relative to theoretical attenuation through a long short-term memory network. The model employs a pothole formation prediction system that superimposes and corrects the baseline decay curve output by the physical mechanism sub-model with the deviation prediction value output by the data-driven sub-model, outputting a PCI prediction value for future time series. When the PCI prediction value is below 75 (or the threshold adjusted for critical road sections) and the weather forecast within the prediction time window includes continuous rainfall (no less than 3 consecutive days of rainfall with a daily rainfall of no less than 10 mm) or freeze-thaw cycles (alternating weather conditions with daily minimum temperature below 0℃ and daily maximum temperature above 5℃ for no less than 5 days), the model outputs a pothole formation risk warning for the corresponding road section, including the risk level and the predicted formation time window. For newly constructed road sections with insufficient data accumulation, the model primarily uses the physical mechanism sub-model for prediction, gradually increasing the contribution weight of the data-driven sub-model as data accumulates.

[0045] By integrating a hybrid prediction model that combines physical mechanisms and data-driven approaches, this approach leverages the interpretability and stability of the physical mechanism model based on mechanical theory, enabling it to provide reasonable estimates of degradation trends even in the early service stages with sparse data or in newly constructed road sections. It also utilizes the powerful fitting capability of the data-driven model, based on long short-term memory networks, for complex nonlinear relationships, allowing the model to accurately capture abnormal degradation patterns in the early stages of pavement performance decline. Compared to pure physical mechanism models or pure data-driven models, this approach offers higher prediction accuracy and stronger generalization ability. Field verification of material parameters in service road sections (core sampling or FWD testing) ensures that the input parameters of the physical mechanism sub-model are consistent with the actual pavement conditions, avoiding prediction errors caused by deviations between design and actual values. When the PCI prediction value is below 75 and accompanied by unfavorable weather conditions, an automatic pothole formation risk warning is triggered, providing an early warning of pothole formation and a sufficient time window for preventative maintenance. The phased activation strategy for newly constructed road sections ensures that the model provides effective predictions at different data accumulation stages, demonstrating good engineering adaptability and incremental optimization capabilities.

[0046] The preset first threshold is a road surface condition index of less than 75, and the risk level is divided into three levels: high risk, medium risk, and low risk, based on the probability of pothole formation. When the probability of pothole formation is greater than or equal to 60%, it is marked as a high-risk level; When the probability of pothole formation is greater than or equal to 30% and less than 60%, it is marked as a medium-risk level; When the probability of pothole formation is less than 30%, it is marked as a low-risk level.

[0047] In one specific implementation, the pothole formation probability is calculated and output by the pothole formation prediction model simultaneously with the PCI prediction value. The specific calculation method is as follows: using the last PCI detection value before pothole formation in historical pothole damage records as a sample, the conditional probability of pothole formation within a subsequent set time window is statistically analyzed under different PCI value ranges. For example, among all road sections with PCI values ​​in the 70-75 range accompanied by rainfall, the proportion of road sections that actually formed potholes within the following 3 months is statistically analyzed. This proportion is the conditional probability of pothole formation under that PCI range and meteorological conditions. A mapping relationship between PCI values ​​and pothole formation probabilities is established (a continuous mapping function from PCI values ​​to probabilities can be established using logistic regression or kernel density estimation methods). When the model predicts that the PCI value of a certain road section will drop to a certain value, the corresponding pothole formation probability is calculated using this mapping function. The risk level classification thresholds (60% and 30%) can be dynamically adjusted based on historical data and maintenance management requirements of the actual road sections. For example, for road sections with particularly high traffic volume (daily traffic volume exceeding 50,000 vehicles) or particularly high safety requirements (such as extra-large bridges and long tunnel sections), the high-risk threshold can be lowered to 50% to improve early warning sensitivity and ensure that key road sections receive higher priority maintenance attention.

[0048] The pothole formation prediction model outputs PCI prediction values ​​and uses the last PCI detection value before pothole formation in historical pothole damage records as a sample. It establishes a continuous mapping function between PCI values ​​and pothole formation probabilities through logistic regression or kernel density estimation. When the model predicts that the PCI value of a certain road segment will drop to a certain value, it calculates the corresponding pothole formation probability through this mapping function. Then, it compares the calculated pothole formation probability with three preset level thresholds. A probability greater than or equal to 60% (or 50% after adjustment for key road segments) is marked as high risk, a probability greater than or equal to 30% and less than 60% is marked as medium risk, and a probability less than 30% is marked as low risk. The corresponding risk level is then output.

[0049] By quantifying the risk of pothole formation into specific probability values ​​and classifying them into high, medium, and low risk levels, the system provides maintenance managers with an intuitive and actionable basis for decision-making. High-risk road sections require immediate attention and priority maintenance, medium-risk road sections require continuous monitoring and timely intervention, and low-risk road sections can be inspected at a regular frequency. This provides a clear risk priority basis for the allocation of maintenance resources. At the same time, the risk level classification thresholds can be dynamically adjusted based on the historical data of the actual road sections and maintenance management requirements. For critical road sections such as major bridges and long tunnels, stricter warning thresholds can be adopted (the high-risk threshold is lowered to 50%), demonstrating good adaptability and flexibility.

[0050] The inspection task plan assigns different inspection frequencies to road sections with different risk levels, specifically configured as follows: The road sections marked as high-risk are assigned an inspection frequency of once a day; The road sections marked as medium risk are assigned an inspection frequency of once every three days; Road sections marked as low-risk are assigned an inspection frequency of once a week.

[0051] As a specific implementation, the digital twin platform includes a task scheduler. Based on the pothole risk prediction results and the marking level of each road segment, the task scheduler automatically generates an inspection task plan containing the inspection route, inspection sequence, and inspection time window. This plan is then distributed to the inspection robot via a wireless communication network. High-risk road segments are inspected daily, meaning at least one complete data collection is completed for that segment within each day's time window (e.g., between 6:00 AM and 6:00 PM). Medium-risk road segments are inspected every three days, meaning at least one complete data collection is completed for that segment within each three-day cycle. Low-risk road segments are inspected weekly, meaning at least one complete data collection is completed for that segment within each week's time window. The inspection robot's movement speed is set to 1-2 km / h in inspection operation mode, using a low-speed, high-precision mode for data collection. The task scheduler automatically optimizes the inspection route sequence based on the inspection robot's current location, charging status, and remaining battery power, prioritizing high-risk sections near the current location to minimize the robot's travel path and energy consumption. When the inspection robot's battery level drops below 20%, the task scheduler automatically inserts a task to return to the wireless charging station for recharging. After recharging, the remaining inspection tasks resume. Once the task plan is generated, the system automatically pushes it to the maintenance management personnel's terminal device for confirmation. Management personnel can adjust the plan (e.g., canceling or postponing inspections due to severe weather, maintenance work, or other special events). Adjustment instructions are synchronized to the inspection robots via the digital twin platform. For long-distance sections with multiple inspection robots deployed, the task scheduler generates separate inspection task plans for each robot's assigned section and coordinates inspection times among the robots to prevent interference caused by multiple robots operating simultaneously on the same section.

[0052] The digital twin platform marks the risk level of each road segment based on the pothole risk prediction results. The task scheduler automatically generates inspection task plans based on the marked level of each road segment. High-risk road segments are inspected once a day, medium-risk road segments are inspected once every three days, and low-risk road segments are inspected once a week. The inspection task plans are distributed to the inspection robots through the wireless communication network. The task plans are executed after being confirmed by the management personnel. The inspection robots move along the guardrails according to the task plan to perform data collection tasks. The collected data is transmitted back to the digital twin platform in real time to update the prediction model. After the model is updated, the risk prediction results are re-output. The task scheduler dynamically adjusts the subsequent inspection task plans according to the updated risk level.

[0053] By dynamically allocating inspection frequencies based on risk levels, precise deployment of inspection resources is achieved. High-risk road sections receive the highest frequency of inspection attention, enabling intensive monitoring during the critical period before potholes form. Medium-risk road sections maintain moderate attention, while low-risk road sections maintain basic coverage with the lowest frequency. Compared to the traditional approach of using a uniform inspection frequency for the entire road section, this invention's differentiated inspection strategy increases the monitoring density of high-risk road sections by 3 to 7 times with the same investment of inspection resources, significantly improving the utilization efficiency of inspection resources. Simultaneously, the task scheduler automatically optimizes inspection routes and inserts charging tasks based on robot location and battery status, ensuring the continuous and reliable execution of inspection tasks. Avoidance rules for management personnel confirmation and special events such as severe weather ensure the safety and rationality of inspection operations. The task coordination mechanism when multiple robots are deployed in segments avoids mutual interference when multiple robots operate simultaneously on the same road section.

[0054] like Figure 2 As shown, the maintenance funding requirement projection module includes: The maintenance demand identification submodule is used to identify a list of road sections that require maintenance intervention in each month, quarter, and unit year based on the pothole formation risk level and predicted formation time window of each road section in the pothole risk prediction results, and to match the corresponding maintenance measure type for each road section in the list. The maintenance measure types include: preventive maintenance measures for road sections where the predicted pavement condition index will drop to the range of 75 to 80 and potholes have not yet formed; repair maintenance measures for road sections where potholes are predicted to form; and structural overhaul measures for road sections where the predicted pavement condition index will drop to below 60. The engineering quantity estimation submodule is used to estimate the implementation engineering quantity of each maintenance measure based on the pavement structure parameters and defect prediction information of each road segment in the road segment list, combined with the preset unit engineering quantity index. The implementation engineering quantity includes the pothole repair area, crack sealing length and preventive maintenance coverage area. The dynamic unit price database module is used to store the unit prices of materials, labor, and machinery shifts for various maintenance measures, and dynamically updates the unit prices of materials, labor, and machinery shifts according to a preset cycle. The funding projection submodule is used to project funding requirements under different maintenance strategies based on the implemented project quantity and the current unit price data in the dynamic unit price database module, using a multi-scenario analysis method.

[0055] As a specific implementation method, the preventive maintenance measures include micro-surfacing, slurry seal, crack sealing, and fog seal, which are suitable for road sections where the PCI predicted value will drop to the 75-80 range but no obvious potholes have yet appeared. Implementing preventive maintenance at this stage can save 5 to 10 yuan in subsequent major repair costs for every 1 yuan invested in preventive maintenance, and can extend the service life of the road surface by 5 to 8 years. The repair maintenance measures include cold patch repair and hot patch repair, which are suitable for road sections where potholes are predicted to form (PCI predicted value will drop to the 70-75 range and pothole formation warning has been triggered). The structural major repair measures include milling and repaving and overlay, which are suitable for road sections where the PCI predicted value will drop below 60 and structural damage has occurred. The preset unit engineering quantity index is determined based on the statistical analysis of historical maintenance engineering quantities. For example, according to historical data, the average repair area of ​​each pothole is 2.5 square meters, the average amount of crack filling material used per linear meter of crack is 0.5 kg, and the micro-surfacing coverage area per lane per kilometer is 3000 square meters. It can be adjusted according to the actual situation of different road sections (for road sections with a high proportion of heavy traffic, the average pothole repair area can be increased to 3.0 square meters). The dynamic unit price database module automatically retrieves the latest material prices, labor prices, and machinery shift prices from market information platforms or through manual input on a monthly or quarterly basis, and records historical price changes for price trend prediction. In the initial stage of system implementation, unit price data is manually entered by maintenance management units to establish an initial database. After the system stabilizes, it connects to the cost information platform of the local public resource trading center or transportation authority via API interface to achieve automatic acquisition and updating of unit prices. Material prices include the unit price of asphalt mixture (yuan / ton), cold patch material (yuan / square meter), and joint sealant (yuan / kg), etc. Labor unit prices include the labor costs of various maintenance trades (yuan / man-day), and machinery shift prices include the shift costs of maintenance machinery such as pavers and rollers (yuan / shift). The funding projection submodule employs a multi-scenario analysis method that includes at least three preset scenarios (aggressive prevention strategy, conventional repair strategy, and conservative strategy), and can accept user-defined additional scenario parameters.

[0056] The maintenance needs identification submodule identifies road sections requiring maintenance intervention in future months, quarters, and years based on the risk level and predicted formation time window of each road segment in the pothole risk prediction results. It then matches the corresponding maintenance measure type based on the PCI prediction value of each road segment: preventative maintenance measures are matched if the PCI is expected to drop to 75-80; repair maintenance measures are matched if potholes are predicted to form (PCI will drop to 70-75); and structural overhaul measures are matched if the PCI drops below 60. The engineering quantity estimation submodule estimates the pavement structure parameters of each road segment in the road segment list and... The system uses disease prediction information, combined with preset unit project quantity indicators (such as an average repair area of ​​2.5 square meters per pothole), to estimate the implementation quantity of various maintenance measures; the dynamic unit price database module establishes an initial database through manual input during the initial stage of system implementation, and after stable operation, it automatically updates the material unit price, labor unit price, and machinery shift unit price of various maintenance measures monthly or quarterly through the API interface of the public resource trading center or cost information platform; the funding projection submodule uses multi-scenario analysis to project the funding requirements under different maintenance strategies based on the implemented project quantity and current unit price data.

[0057] Through the collaborative work of four sub-modules—maintenance needs identification, engineering quantity estimation, dynamic unit price database, and funding projection—a complete automated process from pothole risk prediction to maintenance funding needs projection has been achieved. The maintenance needs identification sub-module transforms technical prediction results into a list of maintenance measures. The engineering quantity estimation sub-module transforms qualitative needs into quantitative engineering quantities (unit indicators based on historical statistical data). The dynamic unit price database module ensures the timeliness and accuracy of unit price data through a two-stage implementation path: manual input followed by automatic API interface integration. The funding projection sub-module integrates engineering quantities and unit prices into operable funding needs data, making maintenance funding needs projection fully traceable. Every budget requirement can be traced back to specific predicted defects, maintenance measures, and engineering quantity estimates, providing a scientific, transparent, and verifiable quantitative basis for maintenance budget preparation.

[0058] The multi-scenario analysis method executed by the capital projection submodule includes: Three maintenance strategy scenarios are set up: aggressive prevention strategy, conventional repair strategy, and conservative strategy. The aggressive prevention strategy is to implement preventive maintenance measures immediately before the predicted formation time window for all road sections marked as high-risk and medium-risk. The conventional repair strategy is as follows: only repair and maintenance measures are implemented on road sections marked as high-risk, while road sections of medium risk are not intervened and are monitored. The conservative strategy is to not implement preventive intervention on any road sections marked as high-risk or medium-risk, and to wait for potholes to actually form before implementing repair maintenance. The funding projection submodule calculates the maintenance funding needs within a future set time period under each maintenance strategy scenario, and generates monthly, quarterly, and annual funding demand distribution data for each scenario. The funding projection submodule also performs a life-cycle cost comparison analysis on each maintenance strategy scenario, calculates the cumulative maintenance cost under each scenario within the preset life-cycle period, and outputs the maintenance strategy scenario with the lowest cumulative maintenance cost as the recommended maintenance plan.

[0059] As a specific implementation method, the aggressive prevention strategy aims to maintain pavement performance at a high level through early intervention. For all road sections marked as high-risk (pothole formation probability ≥60%) and medium-risk (pothole formation probability ≥30% and <60%), preventative maintenance measures such as micro-surfacing and crack sealing are implemented before their predicted formation time window (usually 1 to 2 months in advance) to prevent the eventual formation of potholes. The conventional repair strategy, currently adopted by most maintenance management units, only implements repair maintenance on high-risk road sections (pothole formation probability ≥60%) near their predicted formation time window, while only monitoring is maintained on medium-risk road sections without proactive intervention. The conservative strategy is the most passive maintenance strategy, implementing no preventative intervention on any high-risk or medium-risk road sections, waiting for potholes to actually form and for the damage to fully manifest before implementing repair maintenance. This strategy typically results in the highest cost per maintenance session and the worst pavement performance recovery. The future timeframes are defined as 1 year, 3 years, and 5 years. The preset full life cycle period is 10 to 20 years, with the specific value determined based on the design service life of the road surface. The full life cycle cost comparison analysis includes not only the direct costs of maintenance projects (material costs, labor costs, and machinery costs), but also indirect costs such as traffic delay costs and vehicle detour costs caused by maintenance work. The estimation method for indirect costs is as follows: based on the lane closure plan during maintenance work (e.g., closing one or two lanes), the average daily traffic volume of the road segment (distinguishing between weekdays and holidays), the average single delay time (calculated using traffic wave theory or microscopic traffic simulation software), and the unit time value (based on the regional average wage level and time value conversion factor), the daily traffic delay cost is calculated, multiplied by the estimated number of construction days, to obtain the total traffic delay cost, thus comprehensively reflecting the true economic efficiency of different maintenance strategies. The specific decision-making rule for "outputting the maintenance strategy scenario with the lowest cumulative maintenance cost as the recommended maintenance plan" is as follows: Based on the comparative analysis of the total life cycle cost, the system comprehensively considers the safety risk level (exposure duration and accident probability of high-risk road sections during the waiting period for maintenance) and pavement service level (weighted average of pavement PCI under each strategy). When a certain strategy has the lowest total life cycle cost but a "high" safety risk level, the system automatically recommends the strategy with the second lowest cost but a "medium" or "low" safety risk level. This decision-making logic is marked in the budget report as a note, and the final recommended plan is reviewed and confirmed by the maintenance management personnel.

[0060] The funding projection submodule first sets up three maintenance scenarios: aggressive prevention strategy, conventional repair strategy, and conservative strategy, and defines the triggering conditions and measure types for maintenance intervention in each scenario. Then, based on the pothole risk prediction results and maintenance demand identification results for each road segment, it calculates the required maintenance workload and corresponding funding needs within the future set time period (1 year, 3 years, 5 years) under each scenario, generating monthly, quarterly, and annual funding demand distribution data for each scenario. Finally, it conducts a life-cycle cost comparison analysis for each maintenance strategy scenario. Direct costs (materials, labor, machinery) are calculated based on the workload and unit price, while indirect costs (traffic delays, vehicle detours) are estimated based on lane closure plans, average daily traffic volume, and unit time value during maintenance construction. It calculates the cumulative maintenance cost for each scenario within the preset life-cycle period (10 to 20 years) and outputs recommended maintenance plans based on the decision-making rule of "cost priority, comprehensive balance between safety and service level".

[0061] Through multi-scenario analysis and life-cycle cost comparison, maintenance managers are provided with a comprehensive overview of funding needs and long-term economic comparison data under different maintenance strategies. This upgrades maintenance decision-making from a single-path selection based on managerial experience to optimal strategy selection based on quantitative data. The quantitative introduction of indirect costs (traffic delay costs, vehicle detour costs) makes the comparison of life-cycle costs more comprehensive and realistic, avoiding the bias of underestimating the social costs of conservative strategies when only considering direct engineering costs. Managers can intuitively see that although aggressive prevention strategies have higher short-term capital investment, they have the lowest long-term cumulative costs, while conventional repair strategies have the lowest medium-term costs. While the initial cost of a conservative strategy may seem the lowest, its long-term cumulative cost is higher than that of an aggressive preventative strategy. Conservative strategies, on the other hand, may seem to have the lowest initial cost, but their long-term cumulative cost is the highest (and they also pose the greatest safety risk). This quantitative comparison helps to dispel the misconception that "saving money means less maintenance" and promotes a shift in maintenance management from "passive repair" to "proactive prevention." The multi-dimensional decision-making rules (cost + safety + service level) ensure that the recommended solutions achieve a balance between economy and safety, rather than simply pursuing the lowest cost, which meets the rigid requirements for safety in actual maintenance management. At the same time, the monthly, quarterly, and annual distribution data of funding needs provide a direct basis for the accurate allocation and timing of maintenance funds.

[0062] like Figure 3 As shown, to further implement the present invention, the present invention also provides a method for inspecting potholes on highways, comprising the following steps: Step S1: The digital twin platform generates a daily inspection task plan based on the latest pothole risk prediction results output by the pothole formation prediction model. In the inspection task plan, high-risk road sections, medium-risk road sections and low-risk road sections are assigned different inspection frequencies. Step S2: The inspection robot moves along the highway guardrail according to the inspection task plan. During the movement, the multi-type sensors synchronously collect multi-source perception data of the road surface. The multi-source perception data includes at least visible light images of the road surface, three-dimensional point cloud data of the road surface, motion posture data, real-time position information and vibration signals. Step S3: The edge computing module performs real-time preprocessing on the multi-source sensing data. The preprocessing includes image denoising and enhancement, point cloud filtering and registration, and vibration signal feature extraction. The preprocessed data is uploaded to the digital twin platform through a wireless communication network. Step S4: The data fusion module of the digital twin platform receives the preprocessed multi-source sensing data, aligns and fuses the multi-source sensing data with the historical data in the historical database module on a unified spatial coordinate and time axis, and updates the standard training dataset; Step S5: The model training module retrains or corrects the parameters of the pothole formation prediction model based on the updated standard training dataset, and the updated pothole formation prediction model outputs the updated pothole risk prediction result. Step S6: The maintenance funding demand projection module automatically identifies the maintenance demand type and corresponding maintenance work volume of each target road section based on the updated pothole risk prediction results. Combining the current unit price data in the dynamic unit price database module, it uses a multi-scenario analysis method to project the maintenance funding demand curve under different maintenance strategies and outputs the maintenance funding demand projection results including multi-scenario comparative analysis. Step S7: The digital twin platform automatically generates a multi-dimensional maintenance budget report based on the maintenance funding demand projection results. The multi-dimensional maintenance budget report includes the distribution of funding demand by time dimension, the details of funding demand by road segment dimension, the proportion of funding demand by maintenance measures dimension, and the comparison data of funding demand for different maintenance strategies. It also outputs the maintenance strategy with the lowest cumulative maintenance cost as the recommended maintenance plan. Step S8: Obtain actual maintenance data after the maintenance operation is carried out. The actual maintenance data includes actual maintenance workload, actual maintenance cost, pavement performance test value after maintenance, and maintenance operation quality evaluation. Input the actual maintenance data into the feedback correction module of the digital twin platform. Step S9: The feedback correction module compares the actual maintenance data with the predicted data in the maintenance funding demand projection results, and corrects and updates the model parameters of the pothole formation prediction model according to the deviation comparison results, thus completing the complete closed loop of data collection, pothole prediction, maintenance funding projection and feedback correction.

[0063] As a specific implementation method, the daily inspection task plan in step S1 is generated at midnight (e.g., 5:00 AM) every day, based on the pothole risk prediction results updated the previous day. High-risk road sections are assigned a frequency of once a day, medium-risk road sections are assigned a frequency of once every three days, and low-risk road sections are assigned a frequency of once a week. After the task plan is generated, it is automatically pushed to the management personnel terminal for confirmation. In case of severe weather (such as heavy rain, heavy snow, or dense fog with visibility less than 50 meters) or special events such as roadside construction, the management personnel can cancel or postpone the daily inspection plan with one click. The inspection robot in step S2 moves at a speed of 1~2 km / h. The acquisition actions of all sensors are triggered uniformly by the main control unit of the inspection robot to ensure that all sensors start collecting data at the same time. For long-distance road sections with multiple robots deployed, each robot executes its task plan in parallel according to its respective section. The preprocessed data described in step S3 is uploaded via a 4G or 5G wireless communication network using a streaming transmission method that collects and uploads data simultaneously. When the network is interrupted, the data is temporarily stored locally (with a capacity of no less than 128GB), and resumed transmission when the network is restored. The data fusion module in step S4 uses an improved iterative nearest-point algorithm for spatial alignment (registration accuracy better than 2cm) and Kalman filtering combined with linear interpolation for time synchronization (synchronization accuracy better than 10ms). The retraining or parameter calibration in step S5 employs a combined strategy of "full retraining during off-peak hours daily + rapid incremental fine-tuning after each new data access." During the idle period of computing resources in the early morning (e.g., 2:00 to 5:00), a complete model retraining is performed using all currently accumulated standard training datasets. After each new data access, the current model parameters are fine-tuned using the new data as a validation set; that is, only the weight parameters of the last layer of the LSTM network are updated, rather than completely retraining all network layers, to balance model accuracy and computational resource consumption. The three maintenance strategies set by the multi-scenario analysis method in step S6 are aggressive prevention, conventional repair, and conservative strategies. The future timeframes for the funding demand projection are 1 year, 3 years, and 5 years. The multi-dimensional maintenance budget report in step S7 is automatically generated in electronic document format (PDF or Excel) and pushed to the terminal devices of maintenance management personnel via email or mobile application. The method for obtaining actual maintenance data after the implementation of maintenance work in step S8 is as follows: after completing the maintenance work, the maintenance team enters the actual workload, actual cost, and work quality evaluation through a mobile terminal, and also enters the pavement performance test data (collected by a post-maintenance testing vehicle or portable testing equipment) into the system.The deviation comparison in step S9 includes engineering quantity deviation (the difference between the actual engineering quantity and the predicted engineering quantity), cost deviation (the difference between the actual cost and the predicted cost), and effect deviation (the difference between the actual PCI recovery value and the predicted PCI recovery value after maintenance). Based on the comprehensive analysis results of the three types of deviations, the feedback correction module adopts an incremental learning approach, adding the actual maintenance data as new labeled samples to the training set to fine-tune the LSTM data-driven sub-model. At the same time, it performs Bayesian updates on the material attenuation coefficient in the physical mechanism sub-model based on the actual PCI recovery value, thereby achieving continuous optimization of the pothole formation prediction model.

[0064] This method operates on a daily cycle. Every morning, the digital twin platform generates a daily inspection task plan based on the latest pothole risk prediction results (which is then executed after confirmation by management). The inspection robot moves along the guardrail according to the task plan and simultaneously collects multi-source sensing data (moving speed 1-2 km / h, triggered uniformly by the multi-sensor main control unit). The edge computing module preprocesses the collected data in real time and then uploads it to the digital twin platform via a 4G / 5G wireless network in a streaming manner (local caching in case of network interruption). The data fusion module merges new data with historical data to update the standard training dataset (spatial registration better than 2cm, time synchronization better than 10ms). The model training module uses a combination strategy of "full retraining every morning + incremental fine-tuning after new data access" to update the pothole formation prediction model and output the updated model. The prediction results and maintenance funding demand projection module automatically identify maintenance needs and project funding needs under three strategies (1 year, 3 years, and 5 years) based on the updated prediction results. The digital twin platform automatically generates a multi-dimensional maintenance budget report (PDF / Excel format) and pushes it to management personnel. After the maintenance work is implemented, the actual maintenance data (work volume, cost, PCI recovery value, quality evaluation) is entered into the system through mobile terminals and fed back to the feedback correction module. The feedback correction module compares the actual data with the predicted data in three dimensions: work volume deviation, cost deviation, and effect deviation. It fine-tunes the LSTM model through incremental learning and corrects the parameters of the physical mechanism model through Bayesian updates, thus forming a complete closed loop of "data collection → pothole prediction → maintenance funding projection → feedback correction" that runs on a daily cycle.

[0065] Through a complete nine-step process design, the entire chain of highway pavement pothole inspection—from data collection to maintenance decision-making and effect feedback—is automated and managed in a closed loop. Steps S1 to S3 achieve high-frequency, low-cost automatic collection and real-time uploading of pavement data (streaming transmission with simultaneous collection and uploading, coupled with a local offline caching mechanism, ensures no data loss), solving the problems of low frequency and poor timeliness in traditional inspections. Steps S4 to S5 achieve automatic fusion of multi-source data (sub-centimeter spatial registration and millisecond-level time synchronization) and combined strategy updates for the prediction model (full retraining ensures model accuracy, incremental fine-tuning ensures response speed), enabling prediction accuracy to continuously improve with the extension of running time. Step S Steps 6 to S7 automate the transformation from technical forecasting to financial decision-making, providing a scientific, transparent, and traceable quantitative basis for maintenance budget preparation. Steps S8 to S9 enable automatic feedback and model correction of maintenance effects (three-dimensional comparison of engineering quantity deviation, cost deviation, and effect deviation, with LSTM fine-tuning and physical mechanism Bayesian updates running in parallel), forming a self-reinforcing positive cycle. The management personnel confirmation process and severe weather avoidance rules ensure the safety and rationality of inspection operations. The support for multi-robot segmented deployment makes the method applicable to long-distance highways. The entire process can run automatically without much human intervention, significantly improving the intelligence level and decision-making efficiency of highway pavement maintenance management.

[0066] As a specific implementation method, the digital twin platform also has a decision output interface module, which is used to encapsulate the recommended maintenance plan and maintenance budget report output by the maintenance funding demand projection module into decision instructions that can be parsed by external systems according to a preset standardized data format, and output them to the maintenance operation management platform or automated construction equipment through standardized communication protocols (such as HTTPS protocol, MQTT protocol).

[0067] Specifically, the decision instruction includes at least the following information: the location of the road section requiring maintenance (road section number, starting and ending mileage markers, driving direction and lane information), the type and code of maintenance measures (such as crack sealing, pothole hot patching, micro-surfacing, milling and repaving, etc.), the suggested construction time window (the best construction period generated based on traffic flow forecast and weather forecast), the estimated workload and estimated maintenance cost, and the priority identifier generated according to the pothole risk level.

[0068] After receiving a decision-making instruction, the maintenance operation management platform automatically assigns the maintenance task to the maintenance team closest to the target road section and pushes the maintenance plan, bill of quantities, and construction time window to the team's mobile terminal, generating an electronic work order to enter the construction process. For road sections equipped with automated construction equipment, decision-making instructions can be directly issued to the control systems of equipment such as unmanned pavers and automatic crack sealing machines, allowing the equipment to autonomously execute the maintenance work. After construction is completed, execution data such as actual workload, actual cost, and construction quality parameters are transmitted back to the feedback and correction module of the digital twin platform through the same interface, thus completing a complete closed loop of "prediction → simulation → decision-making → execution → feedback".

[0069] For example, when the digital twin platform, after simulation, determines that preventative micro-surfacing measures should be taken for the high-risk section at K102+350, with a suggested construction window of August 1st to 5th, 2026, an estimated workload of 1200 m², and an estimated maintenance cost of 50,000 yuan, the decision output interface module automatically encapsulates this information into a standard format and pushes it to the maintenance management platform. Upon receiving the instruction, the maintenance management platform automatically dispatches the nearest work team within 10 seconds, allowing the team to begin construction the following day. After construction is completed, data such as actual cost, actual workload, and post-construction PCI recovery value are automatically transmitted back and incorporated into model calibration.

[0070] The decision output interface module adopts a publish-subscribe or request-response communication mode, which transforms the maintenance decision results generated by the digital twin platform from reports that are only for human reading into structured instructions that can be parsed by machines. This enables the maintenance plan to be automatically received, parsed and executed by the downstream system, shortening the time cycle from plan approval to on-site construction, while realizing the quantitative tracking of decision effects.

[0071] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention (including the claims) is limited to these examples; within the framework of the invention, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of the different aspects of the invention as described above, which are not provided in the details for the sake of brevity.

[0072] This invention is intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A highway road surface pothole inspection device, characterized in that, include: The inspection robot sits atop the highway guardrail and moves back and forth along it. The inspection robot is equipped with multiple types of sensors and an edge computing module. The multiple types of sensors are used to synchronously collect multi-source perception data of the road surface during the movement, and the edge computing module is used to perform real-time preprocessing of the multi-source perception data. The digital twin platform is connected to the inspection robot via a wireless communication network and receives the preprocessed multi-source sensing data. The digital twin platform has built-in geometric model construction module, historical database module, data fusion module, model training module, pothole formation prediction model and maintenance funding demand projection module. The geometric model construction module is used to construct a three-dimensional geometric model of the highway section based on the road section design drawings and high-precision maps; the historical database module is used to integrate and store historical pavement technical condition detection data, historical pothole damage records, historical traffic flow data, historical meteorological data, and pavement structure design parameters; the data fusion module is used to align and fuse the multi-source sensing data with the historical data on a unified spatial coordinate and time axis to form a standard training dataset; the model training module is used to train the pothole formation prediction model based on the standard training dataset; the pothole formation prediction model is used to predict the pothole formation risk probability and prediction formation time window of each target road section within a set time window in the future based on the multi-source sensing data and the historical data, and output the pothole risk prediction result; The maintenance funding demand projection module is used to automatically identify the maintenance demand type and corresponding maintenance workload based on the pothole risk prediction results, and, in conjunction with the dynamically updated maintenance unit price database, project the maintenance funding demand curve under different maintenance strategies, and output the maintenance funding demand projection results including multi-scenario comparative analysis. The digital twin platform is also used to dynamically generate the inspection task plan of the inspection robot based on the pothole risk prediction results, and to allocate different inspection frequencies to road sections with different risk levels.

2. The highway road pothole inspection device according to claim 1, characterized in that, The various types of sensors include: A road surface image acquisition unit, which is a high-resolution industrial camera equipped with a global shutter, is used to continuously acquire visible light images of the road surface during the movement of the inspection robot. The visible light images of the road surface are used to identify existing road potholes and road surface cracks. The lidar unit, which is a multi-line lidar, is used to continuously collect three-dimensional point cloud data of the road surface during the movement of the inspection robot. The three-dimensional point cloud data of the road surface is used to generate a road surface elevation model and measure the three-dimensional dimensions of potholes. An inertial measurement unit is used to collect real-time motion attitude data of the inspection robot during its movement. The motion attitude data includes three-axis acceleration, three-axis angular velocity and heading angle, which is used to correct sensor displacement errors caused by uneven road surface and vehicle traffic. A global positioning system module is used to collect the real-time location information of the inspection robot at each data collection moment, and the real-time location information is used to associate the multi-source sensing data with geospatial location; A vibration acceleration sensor is used to collect vibration signals generated when the inspection robot moves along the guardrail. The vibration signals are used to indirectly assess the smoothness of the road surface. The multi-source sensing data includes the visible light image of the road surface, the three-dimensional point cloud data of the road surface, the motion posture data, the real-time location information, and the vibration signal.

3. The highway road pothole inspection device according to claim 2, characterized in that, The various types of sensors undergo unified spatiotemporal calibration: In terms of spatial calibration, a sensor spatial coordinate system is established with the centroid of the inspection robot as the origin. An improved iterative nearest point algorithm is used to achieve pixel-level spatial registration between the three-dimensional point cloud data of the road surface and the visible light image of the road surface, so that the data collected by each sensor has a unified spatial reference. In terms of time calibration, the clock of the global positioning system module is used as the reference clock, and Kalman filtering is used to achieve time synchronization of data from each sensor, with a time synchronization accuracy better than 10 milliseconds; The visible light image of the road surface, the three-dimensional point cloud data of the road surface, the motion posture data, the real-time location information, and the vibration signal, after spatiotemporal calibration, are fused on a unified spatial coordinate and time axis to form a multi-source perception dataset of the road surface with complete spatiotemporal labels.

4. The highway road pothole inspection device according to claim 2, characterized in that, The real-time preprocessing of the multi-source sensing data by the edge computing module includes: The visible light image of the road surface is subjected to noise reduction and contrast enhancement processing; The road surface 3D point cloud data is filtered and registered to remove outliers; The vibration signal is subjected to bandpass filtering and time-domain feature extraction to extract vibration feature parameters including vibration peak value, vibration root mean square value and vibration dominant frequency; The amount of preprocessed multi-source sensing data is less than 30% of the original collected data. The preprocessed multi-source sensing data is transmitted to the digital twin platform through the wireless communication network.

5. The highway road pothole inspection device according to claim 1, characterized in that, The pit formation prediction model is a hybrid prediction model that integrates physical mechanisms and data-driven approaches, including: The physical mechanism sub-model establishes a benchmark attenuation curve for pavement performance based on the pavement structure mechanics attenuation theory. The benchmark attenuation curve takes the pavement structure layer thickness, material elastic modulus and base bearing capacity as input parameters and outputs the theoretical attenuation trajectory of the pavement condition index under the condition of no external intervention. The data-driven sub-model is constructed using a long short-term memory network. It uses the standard training dataset as training samples, road segment basic information, traffic load characteristics, and climate conditions as input features, and the deviation between the actual road surface condition exponential decay value and the benchmark decay curve as the prediction target. It learns the nonlinear deviation pattern of actual decay relative to theoretical decay. The pothole formation prediction model superimposes and corrects the baseline decay curve output by the physical mechanism sub-model and the deviation prediction value output by the data-driven sub-model, and outputs the road surface condition index prediction value in the future time series. When the predicted value of the road surface condition index is lower than a preset first threshold and the weather forecast within the prediction time window includes continuous rainfall or freeze-thaw cycle conditions, the pothole formation prediction model outputs a pothole formation risk warning for the corresponding road segment. The pothole formation risk warning includes a risk level and a predicted formation time window.

6. The highway road pothole inspection device according to claim 5, characterized in that, The preset first threshold is a road surface condition index of less than 75, and the risk level is divided into three levels: high risk, medium risk, and low risk, based on the probability of pothole formation. When the probability of pothole formation is greater than or equal to 60%, it is marked as a high-risk level; When the probability of pothole formation is greater than or equal to 30% and less than 60%, it is marked as a medium-risk level; When the probability of pothole formation is less than 30%, it is marked as a low-risk level.

7. The highway road pothole inspection device according to claim 1, characterized in that, The inspection task plan assigns different inspection frequencies to road sections with different risk levels, specifically configured as follows: The road sections marked as high-risk are assigned an inspection frequency of once a day; The road sections marked as medium risk are assigned an inspection frequency of once every three days; Road sections marked as low-risk are assigned an inspection frequency of once a week.

8. The highway road pothole inspection device according to claim 1, characterized in that, The maintenance funding requirement simulation module includes: The maintenance demand identification submodule is used to identify a list of road sections that require maintenance intervention in each month, quarter, and unit year based on the pothole formation risk level and predicted formation time window of each road section in the pothole risk prediction results, and to match the corresponding maintenance measure type for each road section in the list. The maintenance measure types include: preventive maintenance measures for road sections where the predicted pavement condition index will drop to the range of 75 to 80 and potholes have not yet formed; repair maintenance measures for road sections where potholes are predicted to form; and structural overhaul measures for road sections where the predicted pavement condition index will drop to below 60. The engineering quantity estimation submodule is used to estimate the implementation engineering quantity of each maintenance measure based on the pavement structure parameters and defect prediction information of each road segment in the road segment list, combined with the preset unit engineering quantity index. The implementation engineering quantity includes the pothole repair area, crack sealing length and preventive maintenance coverage area. The dynamic unit price database module is used to store the unit prices of materials, labor, and machinery shifts for various maintenance measures, and dynamically updates the unit prices of materials, labor, and machinery shifts according to a preset cycle. The funding projection submodule is used to project funding requirements under different maintenance strategies based on the implemented project quantity and the current unit price data in the dynamic unit price database module, using a multi-scenario analysis method.

9. The highway road pothole inspection device according to claim 8, characterized in that, The multi-scenario analysis method executed by the capital projection submodule includes: Three maintenance strategy scenarios are set up: aggressive prevention strategy, conventional repair strategy, and conservative strategy. The aggressive prevention strategy is to implement preventive maintenance measures immediately before the predicted formation time window for all road sections marked as high-risk and medium-risk. The conventional repair strategy is as follows: only repair and maintenance measures are implemented on road sections marked as high-risk, while road sections of medium risk are not intervened and are monitored. The conservative strategy is to not implement preventive intervention on any road sections marked as high-risk or medium-risk, and to wait for potholes to actually form before implementing repair maintenance. The funding projection submodule calculates the maintenance funding needs within a future set time period under each maintenance strategy scenario, and generates monthly, quarterly, and annual funding demand distribution data for each scenario. The funding projection submodule also performs a life-cycle cost comparison analysis on each maintenance strategy scenario, calculates the cumulative maintenance cost under each scenario within the preset life-cycle period, and outputs the maintenance strategy scenario with the lowest cumulative maintenance cost as the recommended maintenance plan.

10. A method for inspecting potholes on highway pavements, characterized in that, The following steps are performed using the highway road pothole inspection device according to any one of claims 1 to 9: Step S1: The digital twin platform generates a daily inspection task plan based on the latest pothole risk prediction results output by the pothole formation prediction model. In the inspection task plan, high-risk road sections, medium-risk road sections and low-risk road sections are assigned different inspection frequencies. Step S2: The inspection robot moves along the highway guardrail according to the inspection task plan. During the movement, the multi-type sensors synchronously collect multi-source perception data of the road surface. The multi-source perception data includes at least visible light images of the road surface, three-dimensional point cloud data of the road surface, motion posture data, real-time position information and vibration signals. Step S3: The edge computing module performs real-time preprocessing on the multi-source sensing data. The preprocessing includes image denoising and enhancement, point cloud filtering and registration, and vibration signal feature extraction. The preprocessed data is uploaded to the digital twin platform through a wireless communication network. Step S4: The data fusion module of the digital twin platform receives the preprocessed multi-source sensing data, aligns and fuses the multi-source sensing data with the historical data in the historical database module on a unified spatial coordinate and time axis, and updates the standard training dataset; Step S5: The model training module retrains or corrects the parameters of the pothole formation prediction model based on the updated standard training dataset, and the updated pothole formation prediction model outputs the updated pothole risk prediction result. Step S6: The maintenance funding demand projection module automatically identifies the maintenance demand type and corresponding maintenance work volume of each target road section based on the updated pothole risk prediction results. Combining the current unit price data in the dynamic unit price database module, it uses a multi-scenario analysis method to project the maintenance funding demand curve under different maintenance strategies and outputs the maintenance funding demand projection results including multi-scenario comparative analysis. Step S7: The digital twin platform automatically generates a multi-dimensional maintenance budget report based on the maintenance funding demand projection results. The multi-dimensional maintenance budget report includes the distribution of funding demand by time dimension, the details of funding demand by road segment dimension, the proportion of funding demand by maintenance measures dimension, and the comparison data of funding demand for different maintenance strategies. It also outputs the maintenance strategy with the lowest cumulative maintenance cost as the recommended maintenance plan. Step S8: Obtain actual maintenance data after the maintenance operation is carried out. The actual maintenance data includes actual maintenance workload, actual maintenance cost, pavement performance test value after maintenance, and maintenance operation quality evaluation. Input the actual maintenance data into the feedback correction module of the digital twin platform. Step S9: The feedback correction module compares the actual maintenance data with the predicted data in the maintenance funding demand projection results, and corrects and updates the model parameters of the pothole formation prediction model according to the deviation comparison results, thus completing the complete closed loop of data collection, pothole prediction, maintenance funding projection and feedback correction.

Images