Road guardrail intelligent inspection and maintenance planning method based on machine learning

Abstract

The invention relates to the technical field of intelligent traffic engineering, and provides a road guardrail intelligent inspection and maintenance planning method based on machine learning, which breaks through the limitation of a traditional single mode in a detection link, fuses laser radar three-dimensional geometrical characteristics and visual texture information, realizes high-precision identification of defects such as microcracks and hidden corrosion, and improves the inspection efficiency. The detection problem in a complex environment is solved, and the problem that a traditional method is high in omission ratio is solved. In the aspect of dynamic planning, a multi-dimensional state space is constructed based on real-time traffic, weather and defect levels, polling path dynamic optimization and multi-vehicle collaborative operation are achieved through a reinforcement learning algorithm, and the current situations that manual planning is low in efficiency and poor in flexibility are changed. In the aspect of maintenance decision, a time sequence data analysis model is used for accurately predicting guardrail maintenance requirements, a scientific maintenance plan is made in combination with an intelligent scheduling algorithm, and efficient utilization of resources is achieved. According to the method, a complete intelligent operation and maintenance closed loop is constructed, and road guardrail operation and maintenance are promoted to be converted from passive low efficiency to active intelligence.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent transportation engineering, and specifically to an intelligent inspection and maintenance planning method for highway guardrails based on machine learning. Background Art

[0002] With the continuous growth of China's highway mileage, highway guardrails, as key facilities for ensuring traffic safety, face many technical challenges in their operation and maintenance management. Traditional highway guardrail operation and maintenance technologies have obvious defects: in terms of detection, manual inspection relies on visual observation, making it difficult to detect subtle defects such as micro-cracks and hidden rust, and is greatly affected by environmental factors such as light and weather; single-camera image recognition technology has a high missed detection rate in complex scenarios such as rain, fog, and shadows, and cannot meet the requirements of high-precision detection. In the inspection planning link, manually formulated routes are mostly based on experience, lacking dynamic perception of real-time traffic and road conditions information, resulting in a high vehicle empty driving rate, low inspection efficiency, and difficulty in coping with sudden traffic control and other situations. At the maintenance decision-making level, the maintenance strategy based on a fixed cycle is both likely to cause resource waste and lead to over-maintenance problems, and may also cause maintenance delays and an increase in safety risks due to the failure to handle potential hidden dangers in a timely manner.

[0003] Although existing technologies have tried to improve to a certain extent, for example, some patents have not solved the problem of multi-modal data fusion accuracy, or use fixed algorithms for path planning and cannot adapt to real-time changing road conditions.

[0004] Therefore, an intelligent inspection and maintenance planning method for highway guardrails based on machine learning is proposed. Through the deep coordination of multi-modal data fusion detection, dynamic path planning, and predictive maintenance, an intelligent operation and maintenance closed-loop is formed, effectively improving the accuracy, efficiency, and scientificity of highway guardrail operation and maintenance. Summary of the Invention

[0005] Technical problems to be solved: Aiming at the deficiencies of the existing technology, the present invention provides an intelligent inspection and maintenance planning method for highway guardrails based on machine learning.

[0006] Technical solutions: To achieve the above-mentioned solution purpose, the present invention provides the following technical solutions: An intelligent inspection and maintenance planning method for highway guardrails based on machine learning, including the following steps: S1 Multi-modal data collection and preprocessing: Use an on-vehicle collection system and an unmanned aerial vehicle (UAV) auxiliary system to obtain multi-modal data of the guardrail; achieve time synchronization of multi-source devices through the IEEE1588 Precision Clock Protocol, and complete camera internal and external parameter calibration using Zhang's calibration method and a checkerboard calibration board; remove outliers from the collected point cloud data based on the 3σ criterion and perform voxel grid downsampling processing, and perform Gaussian filtering denoising and adaptive histogram equalization preprocessing on the image data.

[0007] S2 Multimodal Defect Intelligent Recognition: Use the DBSCAN clustering algorithm to extract 12-dimensional geometric features from the preprocessed point cloud data; improve the YOLOv8 object detection model and embed a cross-modal cross-attention module to fuse two-dimensional image features and three-dimensional point cloud features, and train the model based on a dataset containing 100,000 annotated images to detect guardrail defects; fuse 16-dimensional visual features and 12-dimensional point cloud features to form a 28-dimensional feature vector, and input it into an SVM classifier for defect level evaluation.

[0008] S3 Dynamic Inspection Path Planning: Build a reinforcement learning model with 5-dimensional state variables, train it using the Actor-Critic architecture and the PPO algorithm, and optimize the reward function weights through a multi-objective genetic algorithm to achieve inspection decision optimization; combine real-time cloud data and the Dijkstra algorithm to generate a conflict-free inspection path, and allocate subtasks in a multi-vehicle collaborative scenario according to the principle of decreasing defect density from high to low.

[0009] S4 Predictive Maintenance Planning: Establish an LSTM-Attention time series prediction model, input 15-dimensional time series data, and predict the maintenance requirements for the next 6 months; determine the importance weights of road section geographical locations based on the analytic hierarchy process, optimize the maintenance schedule through a genetic algorithm, and consider the constraints of equipment available time and traffic control periods.

[0010] Preferably, when the vehicle-mounted acquisition system and the drone-assisted system obtain multi-modal data of the guardrail, the ranging accuracy of the 16-line solid-state lidar is ±2 cm, the scanning frequency is 10 Hz, the point cloud density is 50 points / m³, and the noise filtering rate reaches 95%; use a dual-channel 4K industrial camera to synchronously collect images at a frame rate of 20 FPS, with a resolution of 10 cm / pixel and a dynamic range of 14 bits; use an environmental sensor group including an SHT30 temperature and humidity sensor, an AWS320 salt fog sensor, and an MPU-9250 IMU sensor to collect environmental temperature and humidity, salt fog concentration, and vehicle motion state data.

[0011] Preferably, the time synchronization achieves a time deviation of multi-source devices <1 μs and a time consistency error <0.1 ms; the Zhang calibration method uses 10 groups of 300 mm × 300 mm checkerboard images taken at different angles to solve the camera internal parameter matrix, with the focal length error controlled within <0.5% and the distortion coefficient <0.01; the external parameter calibration establishes a transformation matrix between the lidar coordinate system and the camera coordinate system, and verifies through 10 repeated calibrations that the spatial positioning error <5 mm.

[0012] Preferably, the noise filtering rate of the point cloud data denoising reaches 95%, the voxel size of 0.05 m is used for downsampling, and the data volume is reduced by 60%; the 3×3 Gaussian filter is used for the image data denoising, with a standard deviation σ = 1.5, and the adaptive histogram equalization technology is used for contrast enhancement, with a contrast enhancement amplitude of 25%.

[0013] Preferably, the DBSCAN algorithm parameter ε is set to 0.15m, MinPts is set to 10, and the extracted 12-dimensional geometric features include the height, width, and length morphological features of the guardrail, the verticality, curvature, and surface roughness structural features, and the three-dimensional coordinates, normal vectors, and adjacent guardrail spacing spatial features; the proportion of manual annotation in the improved YOLOv8 model dataset is 80%, the proportion of AI-assisted annotation is 20%, the annotation consistency is >98%, Mosaic data augmentation and Gaussian blur processing are adopted, and the inference speed on the NVIDIA Jetson AGX Orin edge device reaches 50FPS, with a single-frame time consumption of 18ms, and mAP@0.5 = 95.2%.

[0014] Preferably, the weights of the reward function are optimized by the multi-objective genetic algorithm, specifically -0.6 times the inspection cost, -0.3 times the traffic congestion delay, 1.2 times the detection of high-risk sections, and 0.5 times the equipment utilization rate, achieving a 40% improvement in comprehensive efficiency; the real-time traffic flow detection accuracy in the state space is ±5%, and the weather level is 0 for sunny and 1-4 for bad weather.

[0015] Preferably, the LSTM-Attention model contains 2 LSTM layers with 128 units each, sets dropout = 0.2, 4-head self-attention layer and fully connected layer. After training, the MAE of the validation set is 0.12 and the RMSE is 0.18; the maintenance requirement probability threshold of 0.7 is determined based on cost-risk analysis, and a work order is triggered when the maintenance benefit reaches 1.2 times the cost.

[0016] Preferably, the improved YOLOv8 model detects 8 types of defects such as cracks, deformations, rust, and bolt losses, outputs the pixel coordinates, confidence levels, and two-dimensional dimensions of the defect positions, the confidence level threshold is ≥0.7, and the two-dimensional dimension detection accuracy is ±2mm; the defect pixel coordinates are converted into three-dimensional coordinates in the lidar coordinate system through the external parameter calibration matrix, with an accuracy of ±1m.

[0017] Preferably, the 15-dimensional time series data includes the defect level, repair type, repair cost, monthly average salt spray concentration, rainfall, average temperature, monthly average traffic volume, proportion of heavy-duty vehicles, service life of the guardrail, and material type in the past 36 months.

[0018] Preferably, the analytic hierarchy process invites 10 experts in the field of traffic engineering to participate in questionnaire scoring, and calculates the consistency ratio CR = 0.08; the population size of the genetic algorithm is set to 100, the crossover probability is 0.8, the mutation probability is 0.05, and it iterates 200 times. The objective function considers constraints such as the available time of the crane from 0:00 to 5:00 every day and the traffic control period from 7:00 to 9:00 in the morning rush hour, etc., to achieve the comprehensive optimization of maintenance cost and construction period.

[0019] Beneficial effects: Compared with the prior art, the present invention provides an intelligent inspection and maintenance planning method for highway guardrails based on machine learning, having the following beneficial effects: 1. In the detection link, the intelligent inspection and maintenance planning method for highway guardrails based on machine learning breaks through the limitations of traditional single-modal, integrates the three-dimensional geometric features of lidar and visual texture information, realizes high-precision identification of defects such as micro-cracks and hidden rust, overcomes the detection difficulties in complex environments, and solves the problem of high missed detection rate of traditional methods; in terms of dynamic planning, a multi-dimensional state space is constructed based on real-time traffic, weather, and defect levels, and the inspection path is dynamically optimized and multi-vehicle collaborative operations are realized through reinforcement learning algorithms, changing the current situation of low efficiency and poor flexibility of manual planning; in terms of maintenance decision-making, a time-series data analysis model is used to accurately predict the maintenance requirements of guardrails, and a scientific maintenance plan is formulated in combination with intelligent scheduling algorithms to achieve efficient utilization of resources. Brief Description of the Drawings

[0020] Figure 1 It is a flow schematic diagram of the present invention. Detailed Embodiments

[0021] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0022] Please refer to Figure 1 , the present invention proposes an intelligent inspection and maintenance planning method for highway guardrails based on machine learning, including the following contents: I. Core Steps S1 Multi-modal Data Acquisition and Preprocessing 1.1 Three-dimensional Hardware Acquisition On-vehicle core device: Select a 16-line solid-state lidar (Velodyne VLP-16) as the core device for three-dimensional data acquisition. Its ranging accuracy reaches ±2 cm, which can accurately obtain the three-dimensional coordinate information of the guardrail surface. The scanning frequency is 10 Hz, which can quickly generate continuous point cloud data. The pitch angle adjustment range is -15° to +15°, adapting to different installation heights and detection perspective requirements. The point cloud density of the real-time generated three-dimensional point cloud data of the guardrail is 50 points / m³, and the noise filtering rate can be increased to 95% through the built-in algorithm, providing a high-quality data basis for subsequent defect analysis.

[0023] The dual-channel 4K industrial camera (Basler acA4112-20gm) synchronously captures the images of the guardrail surface at a frame rate of 20 FPS. The installation height is 1.5 m, and the lens viewing angle is 120°. A shooting layout with a left-right symmetric included angle of 90° is formed to ensure that there are no dead angles on the guardrail surface. Images are collected at a resolution of 10 cm / pixel, and the dynamic range reaches 14 bits, enabling clear capture of the texture and color features of apparent defects such as cracks (minimum width ≥ 0.05 mm) and rust (area ratio ≥ 5%).

[0024] The environmental sensor group integrates professional sensors from different manufacturers. The SHT30 temperature and humidity sensor (Sensirion, Switzerland) can accurately measure the environmental temperature and humidity with an accuracy of ±0.3°C / ±2%RH; the AWS320 salt spray sensor can detect the salt spray concentration at a resolution of 0.1 mg / m³; the MPU-9250 IMU monitors the vehicle motion state in real time with an angular velocity accuracy of ±0.05° / s. These sensor data are used to evaluate the corrosion impact of environmental factors on the guardrail and assist in correcting data deviations caused by vehicle motion.

[0025] Drone assistance system: For complex sections such as bridges and mountainous areas that are difficult to cover by vehicle-mounted equipment, a DJI Matrice 300 RTK drone equipped with a Zenmuse P1 full-frame camera is equipped. The flight height of the drone can be flexibly adjusted between 50 - 100 m, the cruising speed is 15 m / s, the single flight duration is 55 minutes, and the operation radius reaches 5 km. It can quickly reach the target area and supplement the collection of the guardrail top and the guardrail under complex terrain from an aerial perspective to obtain complete guardrail data information.

[0026] 1.2 High-precision spatio-temporal calibration Time synchronization: The IEEE 1588 Precision Clock Protocol is adopted, and through a dedicated hardware timestamp synchronization module, time synchronization of multi-source devices such as lidar, industrial cameras, and environmental sensors is achieved, ensuring that the time deviation of each sensor data is < 1 μs and the time consistency error is < 0.1 ms, guaranteeing the precise alignment of the collected multi-modal data in the time dimension and avoiding data fusion analysis failure due to time asynchronization.

[0027] Spatial calibration: The Zhang's calibration method is used for camera internal parameter calibration. Using 10 groups of checkerboard images taken at different angles (checkerboard size 300 mm × 300 mm), the camera internal parameter matrix is solved through mathematical calculations, controlling the focal length error to < 0.5% and the distortion coefficient to < 0.01, and precisely calibrating the optical parameters of the camera.

[0028] External parameter calibration is based on the checkerboard calibration board. By establishing the transformation matrix (R, t) between the lidar coordinate system and the camera coordinate system, after 10 repeated calibrations and verifications, the spatial positioning error is ensured to be < 5 mm. Based on the pinhole camera model and the principle of rigid body transformation, through the formula , realizing precise mapping from pixel coordinates to three - dimensional coordinates with an accuracy of up to ±1m, laying a spatial foundation for multi - modal data fusion.

[0029] 1.3 Data pre - processing Point cloud processing: Removing outliers based on the 3σ criterion, with a noise filtering rate of 95%; then adopting the voxel grid down - sampling technique, setting the voxel size to 0.05m. While retaining the profile features such as guardrail height, verticality, and curvature, the data volume is reduced by 60%, improving the processing efficiency of subsequent algorithms and reducing the computational complexity of subsequent algorithms.

[0030] Image enhancement: First, perform 3×3 Gaussian filtering (standard deviation σ = 1.5) on the collected images to effectively remove salt - and - pepper noise and smooth the images. Then, through the Contrast Limited Adaptive Histogram Equalization (CLAHE) technique, the image contrast is enhanced by 25%, significantly improving the visibility of defects under complex lighting conditions such as backlighting and shadows, enhancing the image quality, and providing clear image data for subsequent defect recognition.

[0031] S2 Intelligent multi - modal defect recognition 2.1 Three - dimensional point cloud feature extraction Use the DBSCAN clustering algorithm to segment the pre - processed point cloud data, with the parameter ε set to 0.15m and MinPts set to 10. This parameter combination is determined through grid search of 100 groups of measured data, achieving a precise segmentation effect with a contour extraction integrity of over 98% and a noise filtering rate of over 95%. The extracted 12 - dimensional geometric features include: Morphological features: Guardrail height, width, and length; Structural features: Verticality (tilt is determined when the deviation > 5°), curvature (local deformation is determined by mutation judgment), surface roughness; Spatial features: Three - dimensional coordinates, normal vectors, and adjacent guardrail spacing.

[0032] These features provide key basis for the quantitative analysis of guardrail structure defects.

[0033] 2.2 Improved YOLOv8 detection model Network structure: Embed a cross - modal cross - attention module in the neck of the original YOLOv8 network. This module calculates the attention weights (weight range 0 - 1) between two - dimensional image features (such as crack edge gradients, rust area color textures) and three - dimensional point cloud features (such as curvature anomaly areas, surface roughness changes), enhancing the model's ability to associate defect features in complex scenarios (such as vegetation occlusion, road surface reflection), effectively improving the detection accuracy of defects such as rust in complex environments, with a 20% improvement compared to the original model.

[0034] Training Configuration: Construct a dataset containing 100,000 labeled images, with 80% manually labeled and 20% AI-assisted labeled. After cross-verification by two professional labelers, the labeling consistency is >98% to ensure the quality of data labeling. The dataset is divided into training / validation / test sets in the ratio of 8:1:1.

[0035] During the training process, the Mosaic data augmentation technique is adopted, randomly stitching 4 images to enrich the diversity of training sample scenarios; the images are Gaussian blurred with a probability of 0.3 to enhance the model's robustness to noisy images. Finally, the model achieves a high-precision detection of mAP@0.5 = 95.2% on the edge device NVIDIA Jetson AGX Orin, with an inference speed of 50 FPS and a single-frame time consumption of only 18 ms, meeting the real-time detection requirements.

[0036] Defect Types: This model can detect 8 types of defects such as cracks, deformations, rust, and missing bolts, and outputs the defect location (pixel coordinates), confidence (threshold ≥ 0.7), and two-dimensional dimensions (accuracy ±2 mm), providing detailed information for defect assessment.

[0037] 2.3 Feature-Level Fusion and Grade Evaluation Spatial Mapping: Using the extrinsic calibration matrix, the defect pixel coordinates detected in the image are accurately converted into three-dimensional coordinates in the lidar coordinate system to achieve the spatial positioning of defects. For example, the pixel coordinates of a certain crack in the image are (1000, 800), and through the coordinate conversion formula , it can correspond to the three-dimensional coordinates (12.5 m, 0.8 m, 1.2 m) with an accuracy of ±1 m, enabling the unified expression of multi-modal data in space.

[0038] Feature Fusion: The extracted 16-dimensional visual features (color histogram, LBP texture) and 12-dimensional point cloud features (curvature, roughness) are formed into a 28-dimensional feature vector and input into an SVM classifier (RBF kernel, penalty coefficient C = 10, determined by 5-fold cross-validation) for grade classification. The defect grades are defined as follows: Grade 0: No defect or defect less than the detection threshold; Grade 1: Crack 0.05 - 0.1 mm or rust area 5% - 10%; Grade 2: Crack 0.1 - 0.3 mm or rust area 10% - 20% or deformation less than 50 mm; Grade 3: Crack ≥ 0.3 mm or rust area ≥ 20% or deformation ≥ 50 mm.

[0039] This classification system provides a clear quantitative standard for maintenance decisions.

[0040] S3 Dynamic Inspection Route Planning 3.1 Reinforcement Learning Model Construction State and Action Space Definition: The state space contains 5 core variables: : Real-time detection of defect level (level 0 - 3); : Real-time traffic flow detected by microwave radar (vehicles / minute, accuracy ±5%); : Weather level obtained in real-time from the meteorological API (0 for clear, 1 - 4 for bad weather); : Road segment length provided by the electronic map (km, accuracy ±10m); : Average of the last 3 inspection intervals, reflecting the historical inspection frequency. The action space is defined as a discrete operation set for selecting uncovered road segments to achieve dynamic decision-making of the inspection route.

[0041] Reward Function Design: The reward function , where each weight is determined by optimizing through a multi-objective genetic algorithm. In 50 sets of simulation scenarios, with the total mileage, the number of defects found, and the equipment utilization rate as the optimization objectives, after iterative search by the genetic algorithm, the optimal weight combination is obtained, which improves the comprehensive efficiency by 40%. -0.6 times of is used to minimize the inspection cost, -0.3 times of to avoid traffic congestion delays, 1.2 times of to ensure priority detection of high-risk road segments, 0.5 times of to improve equipment utilization rate and achieve multi-objective balanced optimization.

[0042] Network Structure: The Actor-Critic architecture is adopted. The Actor network inputs a 128-dimensional state encoding, passes through 2 fully connected layers of 256 dimensions (using the ReLU activation function), and outputs the probabilities of N actions through the Softmax output layer; the Critic network also inputs a 128-dimensional state encoding, and outputs the value function after passing through 2 fully connected layers of 256 dimensions to evaluate the state value. The PPO algorithm is used for training, the batch size is set to 1024, the update interval is 50 steps / iteration, and it is trained based on the data of a 5000-kilometer simulated road segment (covering 10 typical scenarios such as plains, mountains, and bridges) to enable the model to have good generalization ability.

[0043] 3.2 Real-time Planning and Implementation Data Interaction: The cloud server obtains real-time data, including traffic control information, weather warnings, etc., every 10 minutes through 5G / V2X communication technology to update the status of each road segment , and the data transmission delay < 200ms to ensure that the model conducts path planning based on the latest environmental information.

[0044] Strategy Generation: The model preferentially plans sections with a defect level ≥ 2 or a weather level ≥ 3 according to the real-time status, and generates a conflict-free path through the Dijkstra algorithm. In the multi-vehicle collaborative scenario, subtasks are assigned according to the principle of "defect density from high to low". For example, when 3 vehicles cooperate in operation, each vehicle is responsible for detecting sections with a higher defect density within 100 km, realizing efficient collaborative inspection.

[0045] Effect Verification: Actual tests were carried out on a 500-kilometer section of the G60 Shanghai-Kunming Expressway. After 30 repeated experiments, the results showed that the inspection mileage of this solution was shortened by 25.3% compared with manual planning (from 620 km to 463 km), the empty driving rate decreased from 45% to 21.7%, the single-day effective detection mileage increased from 80 km to 162 km, and the standard deviation of the experimental results was < 5 km. Through the t-test (p < 0.05), it shows that the solution has good stability and effectiveness.

[0046] S4 Predictive Maintenance Planning 4.1 LSTM-Attention Prediction Model Input Feature System The model input contains 15-dimensional time series data: Historical Features: Defect level, repair type, and repair cost in the past 36 months; Environmental Features: Monthly average salt fog concentration, rainfall, and average temperature; Traffic Features: Monthly average traffic volume and proportion of heavy-duty vehicles; Basic Features: Service life of the guardrail and material type. Among them, for every 10 mg / m³ increase in the salt fog concentration, the corrosion risk increases by 15%, reflecting the significant impact of environmental factors on the development of defects.

[0047] Network Structure: The model consists of 2 LSTM layers (each with 128 units, setting dropout = 0.2 to prevent overfitting), 4 self-attention layers, and a fully connected layer. The LSTM layer is used to capture the time-dependent relationship of defect development, the self-attention layer focuses on key influencing factors (such as the guardrail with a service life > 10 years, the maintenance demand weight increases by 40%), and the fully connected layer outputs the maintenance demand probability for the next 6 months. During the training process, the mean squared error (MSE) is used as the loss function, the Adam optimizer is used, the learning rate is set to 0.001, and after 100 training epochs, high-precision prediction indicators of MAE = 0.12 and RMSE = 0.18 are achieved on the validation set.

[0048] Maintenance Decision Mechanism Set the maintenance demand probability threshold to 0.7, which is determined based on cost-risk analysis. When the maintenance benefit reaches 1.2 times the cost, a work order is triggered. For example, the maintenance benefit of a level 3 defect is 1.5 times the cost, ensuring a high cost-performance decision.

[0049] 4.2 Multi-objective Optimization Scheduling Priority Score: The formula is used to calculate the maintenance priority score, where is the importance degree of geographical location (bridge / curved road section , ordinary road section ). This weight is determined by the analytic hierarchy process. 10 experts in the field of traffic engineering are invited to participate in questionnaire scoring. After calculation, the consistency ratio CR = 0.08 < 0.1, ensuring the rationality of weight allocation and enabling high-risk and high-importance road sections to be maintained preferentially.

[0050] Genetic Algorithm Configuration: The population size of the genetic algorithm is set to 100, the crossover probability is 0.8 (using single-point crossover method), the mutation probability is 0.05 (randomly reversing the road section order), and it iterates 200 times. The objective function is . In the optimization process, constraints such as the available time of equipment (e.g., the crane is available from 0:00 to 5:00 every day) and traffic control periods (e.g., construction is prohibited during the morning peak from 7:00 to 9:00) are considered to achieve the comprehensive optimization of maintenance cost and construction period.

[0051] Implementation Effect: In the actual application of a 500-kilometer road section, after one-year operation statistics, the maintenance construction period is shortened by 20.1% compared with the traditional plan (from 15 days to 11.98 days), the resource idle rate is reduced from 45% to 29.7%, and the annual maintenance cost is reduced by 35.2% (from 8.5 million yuan to 5.518 million yuan). Through the t-test (p < 0.01), the significant effect of this algorithm in the optimization of maintenance resource scheduling is verified.

[0052] II. Experimental Data and Implementation Cases 1. Multi-modal Defect Recognition Accuracy Test

[0053] Note: The test road section is the G50 Shanghai-Chongqing Expressway (plain section), including 8 types of defect samples. The data is averaged after 3 independent experiments. The t-test shows that p < 0.01.

[0054] 2. Comparison of Dynamic Route Planning Efficiency

[0055] 3. Verification of Predictive Maintenance Effect

[0056] Example 1: Full-process Application on the G60 Shanghai-Kunming Expressway (Zhejiang Section) Road Section Overview: The total length is 320 km, including 50 bridges and 38 curves. The annual average traffic volume is 15,000 vehicles per day, and the salt fog concentration is 45 mg / m³ (coastal section).

[0057] 1. Multi-modal Data Acquisition Implementation Vehicle-mounted Acquisition: The inspection vehicle operates continuously at a speed of 60 km / h, generating 20,000 point cloud data and 200 frames of 4K images per kilometer. The edge device (NVIDIA Jetson AGX Orin) takes less than 1 second for real-time processing and outputs the defect location (accuracy ±1 m) and level in real time.

[0058] UAV Supplementary Acquisition: For 20 high-pier bridges (height > 30 m), the UAV operates at a flight height of 80 m and a speed of 12 m / s. The single-bridge detection time is 3 minutes. After fusing the supplementary data with the vehicle-mounted data, the detection coverage rate of the bridge guardrail is increased from 75% to 100%.

[0059] 2. Defect Identification and Level Assessment Detection Results: A total of 1,237 various defects were found, including 289 hidden rusts missed by manual inspection (accounting for 23.4%) and 197 micro-cracks of 0.05 - 0.1 mm (undetectable by traditional methods).

[0060] Level Distribution: 721 at level 0 (58.3%), 312 at level 1 (25.2%), 168 at level 2 (13.6%), 36 at level 3 (2.9%). All level 3 defects triggered emergency maintenance within 72 hours.

[0061] 3. Dynamic Route Planning Implementation Strategy Application: The cloud dynamically adjusts the route according to the rainstorm warning (W = 4), skips 3 mountain bends (G = 1.5 and D = 2), and preferentially detects high-defect-density areas in the plain section (the proportion of sections with D ≥ 2 is 18%). The inspection cycle is shortened from 12 days in the traditional plan to 2.5 days.

[0062] Efficiency Improvement: 3 inspection vehicles cooperate in operation. Tasks are assigned through the Dijkstra algorithm. The single-vehicle load balance is increased by 35%, and the equipment utilization rate is increased from 60% to 85%.

[0063] 4. Predictive Maintenance Execution Work Order Generation: The predicted maintenance probability for the section from K50 to K60 (salt spray concentration 60 mg / m³, service life 12 years) is 0.78, triggering a first-level work order. 2 cutting machines and 1 crane are dispatched, and the construction window from 0:00 to 5:00 is utilized to complete the repair of level 3 defects within 72 hours, 48 hours earlier than the traditional process.

[0064] Cost-benefit: The annual maintenance cost for this section is reduced from 1.2 million yuan to 750,000 yuan (a reduction of 37.5%). The response time for handling major safety hazards is shortened from 48 hours to 6 hours, and no traffic accidents have occurred due to the failure of the guardrail.

[0065] Example 2: Special Inspection of Special Sections of Bridges Overview of the section: The approach bridge section of a cross-sea bridge (with a total length of 15 km), the annual salt fog concentration > 80 mg / m³, the proportion of heavy-duty vehicles is 30%, and the service life of the guardrail is 8 years.

[0066] 1. Implementation of Multi-modal Inspection The drone flies close to the guardrail at a height of 50 m to obtain the rust data on the top of the guardrail. Combining with the three-dimensional point cloud of the vehicle-mounted lidar, the rust at the base of the guardrail column is detected (the traditional manual missed inspection rate is 40%), and the positioning accuracy reaches ±0.5 m.

[0067] 2. Effect of Maintenance Planning The prediction model warns of the maintenance needs of this section 45 days in advance (P = 0.82). Through genetic algorithm optimization, the maintenance period is compressed to 7 days (the traditional plan requires 15 days), avoiding the peak traffic period of the Spring Festival travel rush. There is no traffic congestion during the maintenance period, and the equipment idle rate is reduced from 50% to 20%.

[0068] Data Verification and Statistical Analysis 1. Significance test: The data of the improved defect recognition rate is tested by t-test. In sunny scenarios, t = 12.3, and in rainy scenarios, t = 15.6, both meeting the significance level of p < 0.001. The data of the reduced maintenance cost is analyzed by variance analysis, with F value = 28.7 and p < 0.0001, proving that there are significant differences between this solution and the traditional method.

[0069] 2. Robustness test: In the temperature range of -10°C to 40°C and the humidity environment of 5% to 95%, the stability error of sensor data < 3%, and the edge-side processing delay fluctuation < 5 ms. In the multi-vehicle cooperation scenario, when the 5G network packet loss rate < 0.1%, the path planning error < 2%, and the system has strong anti-interference ability.

[0070] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for intelligent inspection and maintenance planning of highway guardrails based on machine learning, characterized by: The following steps are involved: S1 Multimodal data acquisition and preprocessing: Use the vehicle-mounted acquisition system and the drone auxiliary system to obtain guardrail multimodal data; use the IEEE1588 precision clock protocol to achieve multi-source device time synchronization, and use Zhang's calibration method and checkerboard calibration board to complete the camera's internal and external parameter calibration; The collected point cloud data is processed by removing outliers based on the 3σ criterion and downsampling using voxel grids, and the image data is preprocessed by Gaussian filtering denoising and adaptive histogram equalization. S2 multi-modal defect intelligent identification: The DBSCAN clustering algorithm is used to extract 12-dimensional geometric features from the pre-processed point cloud data; Improve the YOLOv8 object detection model and embed a cross-modal cross-attention module, fuse 2D image features with 3D point cloud features, and train the model based on a dataset of 100,000 annotated images to detect guardrail defects; fuse 16-dimensional visual features with 12-dimensional point cloud features to form a 28-dimensional feature vector, which is input into the SVM classifier for defect grade assessment; S3 dynamic inspection path planning: build a reinforcement learning model of 5-dimensional state variables, adopt Actor-Critic architecture and PPO algorithm training, optimize the reward function weight through multi-objective genetic algorithm to achieve inspection decision optimization; combine cloud real-time data with Dijkstra algorithm to generate conflict-free inspection paths, and allocate subtasks from high to low defect density in multi-vehicle collaborative scenarios; S4 Predictive maintenance planning: Establish an LSTM-Attention time series prediction model, input 15-dimensional time series data, and predict maintenance needs in the next 6 months; The importance weight of the geographical location of the road section is determined based on the hierarchical analysis method, and the maintenance scheduling is optimized through the genetic algorithm, taking into account the constraints of equipment available time and traffic control period.

2. The method for intelligent inspection and maintenance planning of highway guardrails based on machine learning according to claim 1 is characterized in that: When the vehicle-mounted acquisition system and the UAV auxiliary system acquire the guardrail multimodal data, the 16-line solid-state laser radar has a ranging accuracy of ±2cm, a scanning frequency of 10Hz, a point cloud density of 50 points / m³, and a noise filtering rate of 95%. A dual-channel 4K industrial camera is used to synchronously acquire images at a frame rate of 20FPS, with a resolution of 10cm / pixel and a dynamic range of 14 bits. An environmental sensor group including an SHT30 temperature and humidity sensor, an AWS320 salt spray sensor, and an MPU-9250IMU sensor is used to collect environmental temperature and humidity, salt spray concentration, and vehicle motion status data.

3. The method for intelligent inspection and maintenance planning of highway guardrails based on machine learning according to claim 1 is characterized in that: The time synchronization achieves a multi-source device time deviation of <1μs and a time consistency error of <0.1ms; the Zhang calibration method uses 10 groups of 300mm×300mm chessboard images taken at different angles to solve the camera intrinsic parameter matrix, and the focal length error is controlled at <0.5%, and the distortion coefficient is <0.01; the external parameter calibration establishes a conversion matrix between the laser radar coordinate system and the camera coordinate system, and verifies that the spatial positioning error is <5mm after 10 repeated calibrations.

4. The method for intelligent inspection and maintenance planning of highway guardrails based on machine learning according to claim 1 is characterized in that: The point cloud data denoising has a noise filtering rate of 95%, and the downsampling adopts a voxel size of 0.05m, reducing the data volume by 60%; the image data denoising adopts 3×3 Gaussian filtering, the standard deviation σ=1.5, and the contrast enhancement adopts adaptive histogram equalization technology, with a contrast enhancement range of 25%.

5. The method for intelligent inspection and maintenance planning of highway guardrails based on machine learning according to claim 1 is characterized in that: The DBSCAN algorithm parameter ε is set to 0.15m, MinPts is set to 10, and the extracted 12-dimensional geometric features include guardrail height, width, and length morphological features, verticality, curvature, and surface roughness structural features, and three-dimensional coordinates, normal vectors, and spatial features of the spacing between adjacent guardrails. The improved YOLOv8 model dataset has 80% manual annotations and 20% AI-assisted annotations, with annotation consistency greater than 98%. Mosaic data enhancement and Gaussian blur processing are used, and the inference speed on the NVIDIA Jetson AGX Orin edge device reaches 50FPS, a single frame takes 18ms, and mAP@0.5=95.2%.

6. The method for intelligent inspection and maintenance planning of highway guardrails based on machine learning according to claim 1 is characterized in that: The reward function weights are optimized through a multi-objective genetic algorithm, specifically -0.6 times the inspection cost, -0.3 times the traffic congestion delay, 1.2 times the high-risk road section detection, and 0.5 times the equipment utilization rate, achieving a 40% improvement in overall efficiency; the real-time traffic flow detection accuracy in the state space is ±5%, and the weather level 0 is sunny, and levels 1-4 are severe weather.

7. The method for intelligent inspection and maintenance planning of highway guardrails based on machine learning according to claim 1 is characterized in that: The LSTM-Attention model includes 2 LSTM layers with 128 units, with dropout set to 0.2, 4 self-attention layers and a fully connected layer. After training, the validation set MAE is 0.12 and RMSE is 0.

18. The maintenance requirement probability threshold of 0.7 is determined based on cost-risk analysis, and a work order is triggered when the maintenance benefit reaches 1.2 times the cost.

8. The method for intelligent inspection and maintenance planning of highway guardrails based on machine learning according to claim 1 is characterized in that: The improved YOLOv8 model detects eight types of defects, including cracks, deformation, rust, and missing bolts, and outputs the pixel coordinates, confidence level, and two-dimensional size of the defect position. The confidence threshold is ≥0.7, and the two-dimensional size detection accuracy is ±2mm. The defect pixel coordinates are converted into three-dimensional coordinates in the lidar coordinate system through an external parameter calibration matrix, with an accuracy of ±1m.

9. The method for intelligent inspection and maintenance planning of highway guardrails based on machine learning according to claim 1 is characterized in that: The 15-dimensional time series data include defect level, repair type, repair cost, average monthly salt spray concentration, rainfall, average temperature, average monthly traffic volume, proportion of heavy-loaded vehicles, service life of guardrails, and material type in the past 36 months.

10. The method for intelligent inspection and maintenance planning of highway guardrails based on machine learning according to claim 1, characterized in that: The analytic hierarchy process invites 10 experts in the field of traffic engineering to participate in the questionnaire scoring, and the calculated consistency ratio CR=0.08; the genetic algorithm population size is set to 100, the crossover probability is 0.8, the mutation probability is 0.05, and the iteration is 200 times. The objective function considers the constraints such as the equipment available time is 0:00-5:00 for the crane every day and the traffic control period is 7:00-9:00 during the morning rush hour, so as to achieve comprehensive optimization of maintenance cost and construction period.

Images