Method for intelligently distributing road maintenance tasks based on road surface monitoring

Abstract

The invention relates to the technical field of road maintenance management, in particular to a road maintenance task intelligent allocation method based on road surface monitoring, which comprises the steps of data collection, data processing, feature extraction, model establishment, task allocation, execution and feedback, according to the method, the pavement damage condition is optimized by adopting a pavement damage detection algorithm based on computer vision, and a semantic segmentation task is added on the basis of realizing a pavement damage condition detection task, so that the pavement damage area, the geometric dimension, the pavement damage position and the pavement damage classification can be calculated more accurately and quickly; the model adopts a multivariate performance index combined prediction model, time sequence association of historical data of multiple indexes and information association among the multiple indexes can be mined, and potential association between different types of climate changes and pavement damage trends is fully captured, so that the future pavement damage condition is monitored and predicted more accurately, and the prediction efficiency is improved. Maintenance strategies can be quickly selected for maintenance tasks, task undertaking parties can be quickly allocated, and the road maintenance task allocation efficiency is improved.

Description

Technical Field

[0001] The present application relates to the technical field of highway maintenance management, and in particular to a method for intelligently allocating highway maintenance tasks based on road surface monitoring. Background Art

[0002] With the continuous construction of the national highway network, the demand for highway maintenance is increasing day by day. Investigations and studies have found that good highway maintenance can greatly reduce the possibility and severity of road traffic accidents. Highway maintenance is becoming an important guarantee for my country's highway safety and people's livelihood development. Timely discovery and maintenance of damaged roads can greatly save highway maintenance costs. With the increasing pressure on road network maintenance, intelligent maintenance management technology can effectively improve maintenance efficiency and overcome the limitations of the past maintenance experience-oriented model, and has broad development prospects.

[0003] At present, the overall intelligence level of the maintenance management technology system is still relatively low, the detection accuracy and predictability of road damage are poor, there are still a large number of empirical models in the key links of the maintenance management technology system, and the maintenance management technology lacks analysis and application of data, resulting in the inability to directly guide maintenance construction reliably, accurately and efficiently. Summary of the invention

[0004] In view of the shortcomings of the existing technology, the present invention proposes a method for intelligently allocating highway maintenance tasks based on road surface monitoring, and makes improvements in the selection of monitoring indicators, the road surface damage detection algorithm, and the construction of a highway maintenance task prediction and allocation model. It can more accurately monitor and predict future road surface damage and improve the efficiency of highway maintenance task allocation.

[0005] To achieve the above object, the present invention provides a method for intelligently allocating highway maintenance tasks based on road surface monitoring, comprising the following steps:

[0006] Step 101, data collection: collecting road surface data of the highway through a road surface monitoring system, including road surface damage, traffic flow, and climate conditions;

[0007] Step 102, data processing: pre-processing the collected road surface data, including data cleaning and format conversion, for subsequent analysis;

[0008] Step 103, feature extraction: extracting key features from the preprocessed road surface data, including road surface damage degree, traffic density, and climate conditions; wherein the road surface damage degree is optimized using a road surface damage detection algorithm based on computer vision;

[0009] Step 104, establishing a model; based on the extracted features, establishing a highway maintenance task prediction and allocation model, the model can comprehensively consider the road surface damage, traffic volume, and climate conditions, and use historical maintenance data as a training set to train the model so that the model can learn the development law of road surface damage and predict future damage risks;

[0010] Step 105, task allocation: generating a highway maintenance task allocation plan based on the model prediction results, the plan includes the road sections that need maintenance, the type of maintenance and the priority level;

[0011] Step 106, execution and feedback: the generated maintenance task allocation plan is sent to the relevant maintenance departments for execution, and feedback information during the execution process is collected to continuously optimize the model.

[0012] Furthermore, in step 101, the road surface damage degree is monitored using a line scanning camera or an infrared camera; the road surface damage conditions include cracks, potholes, and flatness;

[0013] Further, in step 102, data cleaning includes removing noise, outliers and duplicate records;

[0014] Furthermore, in step 103, the road damage detection algorithm based on computer vision adopts an end-to-end convolutional neural network that couples the dual tasks of target detection and semantic segmentation. The end-to-end convolutional neural network uses the YOLO network as the skeleton of the target detection part and the CrackW-Net network as the skeleton of the semantic segmentation part.

[0015] Furthermore, in step 104, the model adopts a multi-performance index joint prediction model, which can mine the time series association of multiple index historical data and the information association between multiple indexes, and realize synchronous high-precision prediction of multiple performance indicators;

[0016] Furthermore, the multivariate performance index joint prediction model in step 104 specifically includes: applying a back propagation neural network and a short-term memory network to construct a multivariate performance index joint prediction model for road surface long-term performance prediction; applying an early stopping strategy and an Adam optimizer with an exponentially decaying learning rate to train the model, and using a real data set from a road surface long-term performance database to verify the prediction model;

[0017] Back propagation neural network, which accepts basic characteristics of different road surfaces as input and learns the corresponding implicit influencing factors;

[0018] Short-term memory networks are used to learn time series features of climate conditions of different lengths; time series and cross-sectional features are then implicitly fused by applying an attention mechanism;

[0019] The two parts of the model work in a mutually coupled manner, and the final output represents the overall output of the back propagation neural network and the short-term memory network;

[0020] During model training, an early stopping strategy and an Adam optimizer with an exponentially decaying learning rate were applied. The decay rate parameter was set to 10, and grid search was used to find the optimal setting for the hyperparameters. A real dataset from a long-term pavement performance database was used to validate the prediction model, and three traditional regression models, a pure back propagation neural network model, and a back propagation neural network and short-term memory network fusion model were selected for comparison in basic feature and time series feature experiments.

[0021] Furthermore, the three traditional regression models are linear regression, gradient boosting decision tree, and extreme gradient boosting regression;

[0022] Further, in step 105, the types of maintenance include daily maintenance, preventive maintenance, repair maintenance, special maintenance and emergency maintenance;

[0023] Compared with the prior art, this application has the following beneficial effects:

[0024] The present invention discloses a method for intelligently allocating highway maintenance tasks based on road surface monitoring, including data collection, data processing, feature extraction, model building, task allocation, execution and feedback. In the feature extraction step, a road surface damage detection algorithm based on computer vision is optimized for the road surface damage. On the basis of realizing the road surface damage detection task, a semantic segmentation task is added to realize the end-to-end convolutional neural network design of dual-task coupling, and the road surface damage recognition, classification and geometric feature calculation are realized simultaneously, so that the road surface damage area and geometric size as well as the road surface damage location and classification can be calculated more accurately and quickly; the model adopts a multivariate performance index joint prediction model, which can mine multiple index historical data The model combines the temporal correlation of data and the information correlation between multiple indicators to achieve synchronous high-precision prediction of multiple indicators. The model fuses the back propagation neural network and the short-term memory network through the attention method. The short-term memory network is used to learn the temporal features, and the back propagation neural network is used to learn the basic features. They are fused through the attention method, so that it can give full play to its own network advantages in predicting road damage, each performing its own duties, and associating and coupling, so as to fully capture the potential correlation between different types of climate change and road damage trends, so as to more accurately monitor and predict future road damage, and quickly select maintenance strategies and assign task undertakers for maintenance tasks, thereby improving the efficiency of highway maintenance task allocation.

[0025] Compared with the prior art, the present invention has the following beneficial technical effects:

[0026] (1) Through the combination of monitoring index selection, pavement damage detection algorithm optimization and highway maintenance task prediction and allocation model construction, a comprehensive and accurate pavement monitoring and maintenance task intelligent allocation method is provided, providing a scientific basis for highway maintenance task allocation management and decision-making;

[0027] (2) Based on the object detection network, the damaged pavement size and damaged length can be calculated through the detection frame, but the damaged width cannot be obtained. The image segmentation network can directly calculate the damaged width, but the damaged length and the road surface size of the damaged area cannot be obtained. The pavement damage detection algorithm of the present invention adopts an end-to-end convolutional neural network that couples the dual tasks of target detection and semantic segmentation. The end-to-end convolutional neural network uses the YOLO network as the skeleton of the target detection part and the CrackW-Net network as the skeleton of the semantic segmentation part. Its performance is better than that of a single-task network, and it has the potential to correct the calibration error of the training data. It can simultaneously realize the recognition, classification and geometric feature calculation of pavement damage, and can more accurately and quickly calculate the pavement damage area and geometric size as well as the pavement damage location and classification;

[0028] (3) The model adopts a multi-performance index joint prediction model, which can mine the time series correlation of historical data of multiple indicators and the information correlation between multiple indicators, and realize the synchronous high-precision prediction of multiple indicators. The model integrates the back propagation neural network and the short-term memory network through the attention method. The short-term memory network is used to learn the time series features, and the back propagation neural network is used to learn the basic features. The model is integrated through the attention method, so that it can give full play to its own network advantages in predicting road damage, each performing its own duties, and associating and coupling, so as to fully capture the potential correlation between different types of climate change and road damage trends, so as to more accurately monitor and predict future road damage. It can quickly select maintenance strategies and assign task undertakers for maintenance tasks, thereby improving the efficiency of highway maintenance task allocation. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other embodiments can be obtained based on these drawings without creative work.

[0030] Figure 1 A flow chart of a method for intelligently allocating highway maintenance tasks based on road surface monitoring according to an embodiment of the present invention is exemplarily shown. DETAILED DESCRIPTION

[0031] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0032] It should be noted that the method for intelligently allocating highway maintenance tasks based on road surface monitoring provided in the embodiment of the present application is generally executed by a server / terminal device. Accordingly, the system for intelligently allocating highway maintenance tasks based on road surface monitoring is generally set in the server / terminal device.

[0033] refer to Figure 1 , the figure shows a flow chart of an embodiment of a method for intelligently allocating highway maintenance tasks based on road surface monitoring of the present application, the method for intelligently allocating highway maintenance tasks based on road surface monitoring comprises the following steps:

[0034] The method for intelligently allocating highway maintenance tasks based on road surface monitoring includes:

[0035] Step 101, data collection: collecting road surface data of the highway through a road surface monitoring system, including road surface damage, traffic flow, and climate conditions;

[0036] Step 102, data processing: pre-processing the collected road surface data, including data cleaning and format conversion, for subsequent analysis;

[0037] Step 103, feature extraction: extracting key features from the preprocessed road surface data, including road surface damage degree, traffic density, and climate conditions; wherein the road surface damage condition is optimized using a road surface damage detection algorithm based on computer vision;

[0038] Step 104, establishing a model; based on the extracted features, establishing a highway maintenance task prediction and allocation model, the model can comprehensively consider the road surface damage, traffic volume, and climate conditions, and use historical maintenance data as a training set to train the model so that the model can learn the development law of road surface damage and predict future damage risks;

[0039] Step 105, task allocation: generating a highway maintenance task allocation plan based on the model prediction results, the plan includes the road sections that need maintenance, the type of maintenance and the priority level;

[0040] Step 106, execution and feedback: the generated maintenance task allocation plan is sent to the relevant maintenance departments for execution, and feedback information during the execution process is collected to continuously optimize the model.

[0041] In step 101, the road surface damage degree is monitored by a line scanning camera or an infrared camera; the road surface damage conditions include cracks, potholes, and flatness;

[0042] In step 102, data cleaning includes removing noise, outliers and duplicate records;

[0043] In step 103, the road damage detection algorithm based on computer vision adopts an end-to-end convolutional neural network that couples the dual tasks of target detection and semantic segmentation. The end-to-end convolutional neural network uses the YOLO network as the skeleton of the target detection part and the CrackW-Net network as the skeleton of the semantic segmentation part.

[0044] The total loss of YOLO-Crack consists of four parts: image segmentation loss (0utput1), detection box position loss (0utput2), detection box classification loss (0utput2) and detection box confidence loss (0utput2). The total loss is calculated according to the following formula:

[0045] Loss({Output),{Ground Truth})=w1·CrossEntropy L (Ground Truth-imagc, Outputl)+w2·GIoU L (Otttput2,Ground Truth_box)+w3·CrossEntropy L (Ground Truth_box,Output2)+w4·Confidence L (Ground Truth_box, Output2),

[0046] Among them, w i (i=1, 2, 3, 4) is the weight of the i-th loss. Since the values ​​of the four loss functions vary greatly in magnitude, after each step of training, the weight of the next step is adaptively allocated according to the magnitude of the previous step loss, and is calculated as:

[0047]

[0048] Among them, loss i is the i-th loss, the image segmentation loss (class 1 loss) and the detection box classification loss (class 2 loss) are calculated by the multi-classification cross entropy function, which is calculated as:

[0049] CrossEntropy L (p, q)=-∑p(x)·log(q(x)),

[0050] Among them, p is the probability distribution of the true value, q is the probability distribution of the predicted classification, and the GloU function is used to calculate the position loss of the detection box (type 2 loss), and its calculation formula is:

[0051]

[0052] Among them, A and B are the real detection box position and the predicted detection box position respectively. C is the minimum bounding rectangle containing boxes A and B. The final confidence loss is calculated as:

[0053]

[0054] Among them, Pr(object) is the probability that the detection box detects the road damage.

[0055] In step 104, the model adopts a multi-performance index joint prediction model, which can mine the time series correlation of multiple index historical data and the information correlation between multiple indexes, and realize the synchronous high-precision prediction of multiple performance indicators;

[0056] The multivariate performance index joint prediction model in step 104 specifically includes: applying a back propagation neural network and a short-term memory network to construct a multivariate performance index joint prediction model for road surface long-term performance prediction; applying an early stopping strategy and an Adam optimizer with an exponentially decaying learning rate to train the model, and using a real data set from a road surface long-term performance database to verify the prediction model;

[0057] Back propagation neural network, which accepts basic characteristics of different road surfaces as input and learns the corresponding implicit influencing factors;

[0058] Short-term memory networks are used to learn time series features of climate conditions of different lengths; time series and cross-sectional features are then implicitly fused by applying an attention mechanism;

[0059] The two parts of the model work in a mutually coupled manner, and the final output represents the overall output of the back propagation neural network and the short-term memory network;

[0060] During model training, an early stopping strategy and an Adam optimizer with an exponentially decaying learning rate were applied. The decay rate parameter was set to 10, and grid search was used to find the optimal setting for the hyperparameters. A real dataset from a long-term pavement performance database was used to validate the prediction model, and three traditional regression models, a pure back propagation neural network model, and a back propagation neural network and short-term memory network fusion model were selected for comparison in basic feature and time series feature experiments.

[0061] The three traditional regression models are linear regression, gradient boosting decision tree, and extreme gradient boosting regression;

[0062] In the verification experiment, the evaluation indicators used are mean square error RMSE and determination coefficient R 2 , the indicator formula is as follows:

[0063]

[0064] where y and represent the true value and the predicted value respectively.

[0065] In step 105, the types of maintenance include daily maintenance, preventive maintenance, repair maintenance, special maintenance and emergency maintenance.

[0066] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0067] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A method for intelligently allocating highway maintenance tasks based on road surface monitoring, characterized in that: The steps include: Step 101, data collection: collecting road surface data of the highway through a road surface monitoring system, including road surface damage, traffic flow, and climate conditions; Step 102, data processing: pre-processing the collected road surface data, including data cleaning and format conversion; Step 103, feature extraction: extracting key features from the preprocessed road surface data, including road surface damage degree, traffic density, and climate conditions; wherein the road surface damage condition is optimized using a road surface damage detection algorithm based on computer vision; Step 104, establishing a model; based on the extracted features, establishing a highway maintenance task prediction and allocation model, the model can comprehensively consider the road surface damage, traffic volume, and climate conditions, and use historical maintenance data as a training set to train the model so that the model can learn the development law of road surface damage and predict future damage risks; Step 105, task allocation: generating a highway maintenance task allocation plan based on the model prediction results, the plan includes the road sections that need maintenance, the type of maintenance and the priority level; Step 106, execution and feedback: the generated maintenance task allocation plan is sent to the relevant maintenance departments for execution, and feedback information during the execution process is collected to continuously optimize the model.

2. The method for intelligently allocating highway maintenance tasks based on road surface monitoring according to claim 1 is characterized in that: In step 101, the road surface damage degree is monitored using a line scanning camera or an infrared camera.

3. The method for intelligently allocating highway maintenance tasks based on road surface monitoring according to claim 2 is characterized in that: In step 101, road surface damage includes cracks, potholes, and flatness.

4. The method for intelligently allocating highway maintenance tasks based on road surface monitoring according to claim 1 is characterized in that: In step 102, data cleaning includes removing noise, outliers and duplicate records.

5. The method for intelligently allocating highway maintenance tasks based on road surface monitoring according to claim 1 is characterized in that: In step 103, the road damage detection algorithm based on computer vision adopts an end-to-end convolutional neural network that couples the dual tasks of target detection and semantic segmentation. The end-to-end convolutional neural network uses the YOLO network as the skeleton of the target detection part and the CrackW-Net network as the skeleton of the semantic segmentation part.

6. The method for intelligently allocating highway maintenance tasks based on road surface monitoring according to claim 5 is characterized in that: The total loss of YOLO-Crack consists of four parts: image segmentation loss (Output1), detection box position loss (Output2), detection box classification loss (Output2) and detection box confidence loss (Output2).

7. The method for intelligently allocating highway maintenance tasks based on road surface monitoring according to claim 1, characterized in that: In step 104, the model adopts a multi-performance indicator joint prediction model, which can mine the time series correlation of multiple indicator historical data and the information correlation between multiple indicators, realize the synchronous high-precision prediction of multiple performance indicators, and realize the synchronous high-precision prediction of multiple indicators.

8. The method for intelligently allocating highway maintenance tasks based on road surface monitoring according to claim 7, characterized in that: The multivariate performance index joint prediction model specifically includes: applying a back propagation neural network and a short-term memory network to construct a multivariate performance index joint prediction model for long-term road performance prediction; applying an early stopping strategy and an Adam optimizer with an exponentially decaying learning rate to train the model, and using a real data set from a long-term road performance database to verify the prediction model.

9. The method for intelligently allocating highway maintenance tasks based on road surface monitoring according to claim 8, characterized in that: The two parts of the model work in a mutually coupled manner, and the final output represents the overall output of the back propagation neural network and the short-term memory network; During model training, an early stopping strategy and an Adam optimizer with an exponentially decaying learning rate were applied. The decay rate parameter was set to 10, and grid search was used to find the optimal setting for the hyperparameters. A real dataset from a long-term pavement performance database was used to validate the prediction model, and three traditional regression models, a pure back propagation neural network model, and a back propagation neural network and short-term memory network fusion model were selected for comparison in basic feature and time series feature experiments. The three traditional regression models are linear regression, gradient boosted decision tree, and extreme gradient boosted regression; In the verification experiment, the evaluation indicators used are mean square error RMSE and determination coefficient R 2 .

10. The method for intelligently allocating highway maintenance tasks based on road surface monitoring according to claim 1, characterized in that: In step 105, the types of maintenance include daily maintenance, preventive maintenance, repair maintenance, special maintenance and emergency maintenance.

Images