Highway asphalt pavement maintenance scheme decision method based on bayesian multi-modal neural network

Abstract

The application discloses a highway asphalt pavement maintenance scheme decision method based on a Bayesian multi-modal neural network, which comprises the following steps: constructing a highway pavement maintenance information matrix; selecting highway pavement maintenance decision indexes; constructing a multi-modal neural network model MNN-EAPMD for highway asphalt pavement maintenance scheme decision; constructing a Bayesian multi-modal neural network decision model BMNN-EAPMD for highway asphalt pavement maintenance scheme decision; and constructing a decision optimization scheme based on a maintenance scheme probability uncertainty type. The application proposes a highway pavement maintenance information matrix construction method, a Bayesian multi-modal neural network model for highway asphalt pavement maintenance scheme decision, and a decision optimization method based on uncertainty, so that the mapping relationship between highway basic data and periodic detection indexes and a highway asphalt pavement maintenance scheme is realized, and technical support can be provided for intelligent highway maintenance scheme decision.

Description

Technical Field

[0001] This invention belongs to the research fields of intelligent transportation and smart highways, specifically involving a decision-making method for highway asphalt pavement maintenance schemes based on Bayesian multimodal neural networks. Background Technology

[0002] Currently, highway pavement maintenance decision-making primarily relies on periodic pavement inspections to obtain pavement condition information, identifying sections requiring preventative and restorative maintenance, and combining this with expert on-site inspections to determine the appropriate maintenance construction plan. However, due to the limitations of periodic pavement inspection reports, actual maintenance construction plans often differ from the design plans presented in these reports. Therefore, this invention proposes a data-driven, artificial intelligence-based maintenance plan decision-making model. This model characterizes the mapping relationship between highway baseline data, periodic inspection indicators, and highway asphalt pavement maintenance plans, thereby achieving intelligent highway maintenance plan decision-making. Summary of the Invention

[0003] Purpose of the invention: In order to overcome the shortcomings of the existing technology, this invention provides a decision-making method for highway asphalt pavement maintenance schemes based on Bayesian multimodal neural networks. It establishes a highway maintenance information matrix and constructs a decision-making method for highway asphalt pavement maintenance schemes based on Bayesian multimodal neural networks, which can provide technical support for intelligent decision-making on highway maintenance schemes.

[0004] Technical Solution: To achieve the above objectives, this invention provides a decision-making method for highway asphalt pavement maintenance schemes based on Bayesian multimodal neural networks, comprising the following steps:

[0005] S1: Construct a highway pavement maintenance information matrix.

[0006] S2: Select decision indicators for highway pavement maintenance.

[0007] S3: Construct a multimodal neural network model MNN-EAPMD for decision-making on highway asphalt pavement maintenance schemes.

[0008] S4: Construct a Bayesian multimodal neural network decision model BMNN-EAPMD for highway asphalt pavement maintenance scheme decision-making.

[0009] S5: Construct a decision optimization scheme based on the probabilistic uncertainty of maintenance schemes.

[0010] Furthermore, the specific method for constructing the highway pavement maintenance information matrix in step S1 is as follows:

[0011] S1-1: Collect highway pavement maintenance data, including descriptions of basic highway characteristics such as technical grade, pavement structure, maintenance unit, start and end station numbers, lanes, year of construction, traffic volume, and meteorological data; descriptions of highway pavement performance such as Pavement Damage Index (PCI), Road Quality Index (RQI), Road Rutting Index (RDI), Road Skid Resistance Index (SRI), Pavement Wear Index (PWI), Road Bump Index (PBI), and Structural Strength Index (PSSI); and descriptions of historical highway maintenance conditions such as the start and end station numbers of defects, defect names, and defect lengths.

[0012] S1-2: Extract, transform, and load highway pavement maintenance data from different sources and in different formats to construct an information matrix.

[0013] S1-3: Preprocess highway pavement maintenance data.

[0014] S1-4: Before training each model, randomly select image data to divide it into training and test sets.

[0015] Furthermore, the specific content of constructing the information matrix in step S1-2 is as follows:

[0016] ① Determine the index data and construct the information table. This invention selects 1 kilometer as the length of the road segment unit, that is, the length of the highway segment of the unit sample. The index data of the data table is set as "starting point chainage - ending point chainage - up / down direction - lane". At the same time, in order to facilitate indexing, the "K...+..." of the starting and ending point chainages are converted into integer values, for example, "K80+300" is converted into "80300".

[0017] ② Manual data entry. For highway pavement maintenance data with small data volume and simple sources, highway pavement maintenance data from different sources can be collected into a single data table by manual entry. This includes data such as climate data, construction time, road age, and pavement structure. Such data usually does not need to be updated or is updated infrequently.

[0018] ③ Batch Loading. For large volumes of highway pavement maintenance data, the data is extracted from various sources and processed through a series of transformations and integrations before being loaded into the database. These sources include periodic inspection reports, construction drawing design tables, detailed damage tables, and traffic volume survey tables. For structured data, data reading and format conversion are performed, and the data is matched against the index data before being loaded into the data table. For unstructured data, character segmentation is used to clarify the meaning of each part. For example, for a construction drawing design table, the location information of a road segment might be in the form of "downward K47+000-K50+000". The string can be segmented into five parts based on "K", "+", and "-", representing the up and down directions, and the start and end point locations, thus finding the corresponding index data. Simultaneously, for descriptions of different damage types and maintenance schemes, such as "transverse crack", "transverse joint", and "lateral fissure", character segmentation and fuzzy matching are used to convert them into a unified description. For example, "transverse and longitudinal joints" are converted into ['transverse crack', 'longitudinal crack'], and then matched against the index data before being loaded into the data table.

[0019] ④ Constructing the information matrix. For each road segment unit, the data for each year is used as a feature vector. The road segment unit data are stacked based on spatial coordinates to form a two-dimensional information matrix. Then, considering the time dimension, a time-series-based three-dimensional information matrix is ​​constructed according to the chronological order of the highways.

[0020] Furthermore, the specific content of the highway pavement maintenance data preprocessing in S1-3 is as follows:

[0021] ①One-hot encoding. Use one-hot encoding for features with few states, such as up / down and lane features. "Up / down, lane one" is converted to [1,0,1,0,0].

[0022] ② Label encoding. First, word segmentation, combination and deduplication are performed on the data of Shaanxi expressway pavement structures and disease types to generate the following vocabulary: {'AC':0, 'zhong':5, 'erhui':6, 'xi':19, 'ATB':1, 'cu':17, 'SMA':3, 'micro-surfacing spalling':11, 'cement':14, 'insufficient pavement anti-skid performance':22, 'crack':21, 'pothole':9, 'SBS':2, 'longitudinal crack':18, 'rutting':23, 'transverse crack':13, 'repair':7, 'swelling':12, 'mud pumping':8, 'dense transverse crack':10, 'subsidence':15, 'crocodile cracking':20, 'burnout':16, 'sinking':4}, then the text data is mapped to numerical data according to the vocabulary. If the number of disease types in different sections of the expressway is different, which results in failure to obtain feature vectors of the same length after conversion, a padding operation is performed to append zeros at the back to align the data. The converted data form is as follows: [1,19,3,17,6,13,18,23,0,0,0,0].

[0023] ③ Missing value processing. In expressway pavement maintenance data, pavement performance data, traffic volume data, climate data and the like are all continuous variables on time series data. If there are missing values, an interpolation method can be used to fill the missing values; for categorical data such as maintenance measures, the missing value can be regarded as a new category; for other null values, a placeholder can be used for filling.

[0024] ④ Outlier detection. The present invention uses an interquartile range (IQR) detection method to process outliers in expressway pavement maintenance data. First, the third quartile Q3, the first quartile Q1 and the median of the data are calculated; second, the box height IQR=Q3-Q1 is calculated; third, the outlier range is determined, with the upper limit being Q3+1.5×IQR and the lower limit being Q1-1.5×IQR; finally, data points beyond this range are regarded as outliers and eliminated.

[0025] ⑤ Data normalization. To avoid the model assigning different weights to features of different magnitudes and relatively weakening the effect of features with small numerical values, max-min normalization is adopted for scaling:

[0026]

[0027] ⑥ Data standardization. To adjust the data distribution to a standard normal distribution for facilitating the use of subsequent machine learning models, the z-Score method is adopted for data standardization, where μ is the mean of all samples, and σ is the standard deviation of all samples:

[0028]

[0029] Furthermore, the specific content of selecting highway pavement maintenance decision indicators in step S2 is as follows:

[0030] S2-1: Highway asphalt pavement maintenance projects are roughly divided into three types: preventive maintenance, repair maintenance, and routine maintenance. Based on the maintenance construction plans recorded in Shaanxi highways in recent years, statistics and simplified classifications are made, and finally, highway asphalt pavement maintenance plans are divided into six categories: "crack sealing", "ultra-thin wear course", "micro-surfacing", "milling and repaving of the top layer", "milling and repaving of the upper and middle layers", and "milling and repaving of the upper, middle and lower layers".

[0031] S2-2: Factors and Feature Selection Influencing Maintenance Scheme Decisions. Considering the main decision-making processes for highway asphalt pavement maintenance in the past and present, the following features were selected: distress length, distress type, pavement age, PCI, mixed equivalent, pavement structure, passenger-to-freight ratio, RDI, construction time, SRI, PQI, RQI, direction of travel, and lane number.

[0032] S2-3: Input the cleaned highway asphalt pavement maintenance data into the decision tree, classify the maintenance schemes based on the data of that year, calculate the impact of each feature on the maintenance scheme decision, and obtain the feature importance ranking.

[0033] Furthermore, the specific content of constructing the multimodal neural network model MNN-EAPMD for highway asphalt pavement maintenance scheme decision-making in step S3 is as follows:

[0034] S3-1: Divide the decision data applied to the highway asphalt pavement maintenance scheme in step S1 into two categories: continuous data and text data (including disease types and pavement structure). Then, encode the text data according to the method in step S1-3 and input it into the decision model.

[0035] S3-2: Set up a Numerical feature extraction subnetwork, take continuous highway pavement maintenance data as input data, use a fully connected layer Dense to extract data features, and use the tanh function for activation.

[0036] S3-3: Set up a FastText feature extraction subnetwork to perform an embedding operation on the input highway pavement maintenance text data, so that each character is converted into a 1×20 embedding vector. After the embedding layer, a GlobalAveragePooling1D layer is connected to average the embedding vectors, and then a Dense hidden layer is connected to activate it using the tanh function. Finally, a classifier is used to obtain the classification result.

[0037] S3-4: Use the feature-based concat method to perform feature fusion on the output of the sub-network.

[0038] S3-5: Connect a Batch Normalization layer to perform batch normalization of parameters, thereby accelerating model convergence.

[0039] S3-6: Connect a Dense layer as a fusion feature extraction layer.

[0040] S3-7: Connect another Dense layer as the output layer, and use softmax as the loss function of the model. Modify the recognition output results to 7 categories, which correspond to the 7 schemes in the maintenance construction scheme library: "No maintenance", "Crack filling", "Ultra-thin wear layer", "Micro-surfacing", "Milling and repaving the top layer", "Milling and repaving the upper and middle layers", and "Milling and repaving the upper, middle and lower layers".

[0041] Furthermore, the specific details of using the feature-based concatenation (Concat) method to perform feature fusion on the output of the sub-network in steps S3-4 are as follows:

[0042] ① Calculate the mean and variance of d neurons:

[0043]

[0044]

[0045] ② The parameters are regularized, and the calculation method is as follows:

[0046]

[0047] ③ A linear transformation is used to shift the data distribution from (0,1) to (-1,1). The calculation method is as follows:

[0048]

[0049] Furthermore, the specific content of constructing the Bayesian multimodal neural network decision model BMNN-EAPMD for highway asphalt pavement maintenance scheme decision-making in step S4 is as follows:

[0050] S4-1: Divide the decision data applied to the highway asphalt pavement maintenance scheme in step S1 into two categories: continuous data and text data (including disease types and pavement structure). Then, encode the text data according to the method in step S1-3 and input it into the decision model.

[0051] S4-2: Set up a Numerical feature extraction subnetwork, take continuous highway pavement maintenance data as input data, use a fully connected layer Dense to extract data features, and use the tanh function for activation.

[0052] S4-3: Set up a FastText feature extraction subnetwork to perform embedding operations on the input highway pavement maintenance text data, so that each character is converted into a 1×20 embedding vector. After the embedding layer, a GlobalAveragePooling1D layer is connected to average the embedding vectors, and then a Dense hidden layer is connected, which is activated by the tanh function. Finally, a classifier is used to obtain the classification result.

[0053] S4-4: Based on MNN-EQPMD, the Tainer and MC Dropout methods are used, specifically: a Dropout layer is inserted after each fully connected layer and before the non-linear activation function.

[0054] S4-5: The outputs of the sub-networks are fused using Concat, with the specific operation method being the same as step S3-3, and the parameters are batch normalized using the Batch Normalization method.

[0055] S4-6: Connect a Dense layer to extract fused features. This Dense layer is also connected using Dropout and then enters the tanh activation function.

[0056] S4-7: Connect another Dense layer as the output layer, and use softmax as the loss function of the model. Modify the recognition output results to 7 categories, which correspond to the 7 schemes in the maintenance construction scheme library: "No maintenance", "Crack filling", "Ultra-thin wear layer", "Micro-surfacing", "Milling and repaving the top layer", "Milling and repaving the upper and middle layers", and "Milling and repaving the upper, middle and lower layers".

[0057] S4-8: Since the output of the model follows a certain probability distribution for each run, the statistical analysis of the results can yield two output indicators: [scheme, probability].

[0058] Furthermore, the specific content of the statistical method for the output results of each run of the model in steps S4-8 is as follows:

[0059] ①The output of the BMNN-EAPMD model for each run is an x-dimensional vector, where x corresponds to the category output by the model. The index of the maximum value in the model output is taken as the predicted category.

[0060] ② Run the test on the test set n times, and statistically analyze the class prediction results of the n predictions. The statistical structure is as follows: [(M i ,p i ),(M k ,p k )……(M m ,p m )], where Mx For the decision category of the BMNN-EAPMD model, p x This represents the probability corresponding to that category.

[0061] ③ The probability of classification in the BMNN-EAPMD model represents the likelihood of a sample belonging to different categories. For an N-class classification problem, the probability of each sample corresponding to each category can also be represented as an N-dimensional vector, with the sum of the probabilities of each category being 1. For the BMNN-EAPMD decision model, if the probability of a sample being classified into class M1 is 0.5 and the probability of being classified into class M2 is 0.4, then the BMNN-EAPMD model cannot accurately determine which category the sample belongs to, indicating that the sample belongs to a marginal case.

[0062] Furthermore, the specific content of constructing the decision optimization scheme based on the probabilistic uncertainty of the maintenance scheme in step S5 is as follows:

[0063] S5-1: Regarding the decision outcome [(M i ,p i ),(M k ,p k )……(M m ,p m Sort the data according to probability from largest to smallest, denoted as [(M] x1 ,p x1 ),(M x2 ,p x2 )……(M xn ,p xn )], where p x1 >p x2 >……>p xn , where n is the number of decision schemes given by BMNN-EAPMD, and n≥1;

[0064] S5-2: If there is p x If the confidence level is greater than 0.6, then the scheme is considered to have a high confidence level, and M is selected. x The plan prioritizes the inclusion of corresponding highway sections in the maintenance schedule.

[0065] S5-3: If 0.4 ≤ p x If the ratio is ≤0.6, then the corresponding maintenance plan is considered feasible for the highway section, and a cost-benefit comparison should be conducted. The objective of the plan comparison is to maximize the benefit-cost ratio r, which is calculated as follows:

[0066]

[0067]

[0068] Where C represents maintenance cost, k1, k2, λ1, and λ2 are coefficients adjusted according to actual conditions (default value is 1), B represents maintenance benefit, State represents benefit evaluation index, t represents the decision year, and s represents the number of benefit evaluation indicators used. For the sake of simplification, this paper uses only PCI as the benefit evaluation index when evaluating maintenance benefit B, and the calculation method is as follows:

[0069] B = ΔPCI = PCI t+1 PCI t

[0070] Among them, PCI t As is known, PCI t+1 The PCI value for the following year is obtained from the prediction model. Different predicted PCI values ​​can be obtained by changing the maintenance scheme type for the decision year in the input data. t+1 Based on the prediction results and maintenance costs, a comparison of different options is conducted.

[0071] S5-4: If all solutions have p x A value <0.4 indicates that the decision-making model cannot make an accurate judgment on this highway section based solely on the current database. The decision categories and corresponding probabilities output by the model should be submitted to the manager to assist in decision-making. The manager should conduct an in-depth analysis of the specific pavement defects of the highway, combining this with a comprehensive analysis of funding and benefits, to determine a maintenance plan and correct the results of the decision-making model.

[0072] Beneficial effects: Compared with the prior art, the present invention realizes the decision-making of highway asphalt pavement maintenance scheme based on deep learning technology, improves the model performance, and can provide technical support for intelligent decision-making of highway maintenance schemes. Attached Figure Description

[0073] Figure 1 Box plots showing the accuracy of each model on the test set.

[0074] Figure 2 This is a data information matrix for highway maintenance.

[0075] Figure 3 This is a schematic diagram of the BMNN-EAPMD model structure. Detailed Implementation

[0076] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments.

[0077] This invention provides a method for predicting the service performance of highway asphalt pavement based on a priori recurrent neural networks, comprising the following steps:

[0078] S1: Construct a highway pavement maintenance information matrix, specifically as follows:

[0079] S1-1: Collect highway pavement maintenance data, including descriptions of basic highway characteristics such as technical grade, pavement structure, maintenance unit, start and end station numbers, lanes, year of construction, traffic volume, and meteorological data; descriptions of highway pavement performance such as Pavement Damage Index (PCI), Road Quality Index (RQI), Road Rutting Index (RDI), Road Skid Resistance Index (SRI), Pavement Wear Index (PWI), Road Bump Index (PBI), and Structural Strength Index (PSSI); and descriptions of historical highway maintenance conditions such as the start and end station numbers of defects, defect names, and defect lengths.

[0080] S1-2: Extract, transform, and load highway pavement maintenance data from different sources and in different formats to construct an information matrix. The resulting decision dataset contains 953 decision data points, including seven types of decisions: "No maintenance," "Crack sealing," "Ultra-thin wearing course," "Micro-surfacing," "Milling and repaving of the top layer," "Milling and repaving of the upper and middle layers," and "Milling and repaving of the upper, middle, and lower layers."

[0081] ① Determine the index data and construct the information table. This invention selects 1 kilometer as the length of the road segment unit, that is, the length of the highway segment of the unit sample. The index data of the data table is set as "starting point chainage - ending point chainage - up / down direction - lane". At the same time, in order to facilitate indexing, the "K...+..." of the starting and ending point chainages are converted into integer values, for example, "K80+300" is converted into "80300".

[0082] ② Manual data entry. For highway pavement maintenance data with small data volume and simple sources, highway pavement maintenance data from different sources can be collected into a single data table by manual entry. This includes data such as climate data, construction time, road age, and pavement structure. Such data usually does not need to be updated or is updated infrequently.

[0083] ③ Batch Loading. For large volumes of highway pavement maintenance data, the data is extracted from various sources and processed through a series of transformations and integrations before being loaded into the database. These sources include periodic inspection reports, construction drawing design tables, detailed damage tables, and traffic volume survey tables. For structured data, data reading and format conversion are performed, and the data is matched against the index data before being loaded into the data table. For unstructured data, character segmentation is used to clarify the meaning of each part. For example, for a construction drawing design table, the location information of a road segment might be in the form of "downward K47+000-K50+000". The string can be segmented into five parts based on "K", "+", and "-", representing the up and down directions, and the start and end point locations, thus finding the corresponding index data. Simultaneously, for descriptions of different damage types and maintenance schemes, such as "transverse crack", "transverse joint", and "lateral fissure", character segmentation and fuzzy matching are used to convert them into a unified description. For example, "transverse and longitudinal joints" are converted into ['transverse crack', 'longitudinal crack'], and then matched against the index data before being loaded into the data table.

[0084] ④ Constructing the information matrix. For each road segment unit, the data for each year is used as a feature vector. The road segment unit data are stacked based on spatial coordinates to form a two-dimensional information matrix. Then, considering the time dimension, a time-series-based three-dimensional information matrix is ​​constructed according to the chronological order of the highways.

[0085] S1-3: Preprocessing of highway pavement maintenance data:

[0086] ①One-hot encoding. Use one-hot encoding for features with few states, such as up / down and lane features. "Up / down, lane one" is converted to [1,0,1,0,0].

[0087] ② Label encoding. First, word segmentation, combination and deduplication are performed on the data of Shaanxi expressway pavement structures and disease types to generate the following vocabulary: {'AC':0, 'Zhong':5, 'Lime-fly ash':6, 'Fine':19, 'ATB':1, 'Coarse':17, 'SMA':3, 'Micro-surfacing spalling':11, 'Cement':14, 'Insufficient pavement anti-skid performance':22, 'Crack':21, 'Pothole':9, 'SBS':2, 'Longitudinal crack':18, 'Rutting':23, 'Transverse crack':13, 'Repair':7, 'Bump':12, 'Pumping':8, 'Dense transverse crack':10, 'Subsidence':15, 'Network crack':20, 'Burnout':16, 'Settlement':4}, then the text data is mapped to numerical data according to the vocabulary. If the number of disease types in different sections of the expressway is different, resulting in failure to obtain feature vectors of the same length after conversion, a padding operation is performed to append 0s at the end to align the data. The converted data form is as follows: [1,19,3,17,6,13,18,23,0,0,0,0].

[0088] ③ Missing value processing. In expressway pavement maintenance data, pavement performance data, traffic volume data, climate data and other data are continuous variables on time series data. If there are missing values, the interpolation method can be used to fill the missing values; for categorical data such as maintenance measures, the missing value can be regarded as a new category; for other null values, placeholders can be used for filling.

[0089] ④ Outlier detection. The present invention uses the interquartile range (IQR) detection method to process outliers in expressway pavement maintenance data. First, the third quartile Q3, the first quartile Q1 and the median of the data are calculated; secondly, the box height IQR = Q3-Q1 is calculated; thirdly, the outlier range is determined, with the upper limit being Q3+1.5×IQR and the lower limit being Q1-1.5×IQR; finally, data points beyond this range are regarded as outliers and eliminated.

[0090] ⑤ Data normalization. To avoid the model assigning different weights to features of different magnitudes and relatively weakening the effect of features with small values, min-max normalization is adopted for scaling:

[0091]

[0092] ⑥ Data standardization. To adjust the data distribution to a standard normal distribution for the convenience of subsequent use of machine learning models, the z-Score method is adopted for data standardization, where μ is the mean of all samples and σ is the standard deviation of all samples:

[0093]

[0094] S1-4: Before training each model, image data is randomly sampled in a ratio of 8:1:1 to divide the training set, validation set, and test set.

[0095] S2: Select decision indicators for highway pavement maintenance, the specific contents of which are as follows:

[0096] S2-1: Highway asphalt pavement maintenance projects are roughly divided into three types: preventive maintenance, repair maintenance, and routine maintenance. Based on the maintenance construction plans recorded in Shaanxi highways in recent years, statistics and simplified classifications are made, and finally, highway asphalt pavement maintenance plans are divided into six categories: "crack sealing", "ultra-thin wear course", "micro-surfacing", "milling and repaving of the top layer", "milling and repaving of the upper and middle layers", and "milling and repaving of the upper, middle and lower layers".

[0097] S2-2: Factors and Feature Selection Influencing Maintenance Scheme Decisions. Considering the main decision-making processes for highway asphalt pavement maintenance in the past and present, the following features were selected: distress length, distress type, pavement age, PCI, mixed equivalent, pavement structure, passenger-to-freight ratio, RDI, construction time, SRI, PQI, RQI, direction of travel, and lane number.

[0098] S2-3: Input the cleaned highway asphalt pavement maintenance data into the decision tree, classify the maintenance schemes based on the data of that year, calculate the impact of each feature on the maintenance scheme decision, and obtain the feature importance ranking.

[0099] S3: Construct a multimodal neural network model MNN-EAPMD for decision-making on highway asphalt pavement maintenance schemes. The specific content is as follows:

[0100] S3-1: Divide the decision data applied to the highway asphalt pavement maintenance scheme in step S1 into two categories: continuous data and text data (including disease types and pavement structure). Then, encode the text data according to the method in step S1-3 and input it into the decision model.

[0101] S3-2: Set up a Numerical feature extraction subnetwork, take continuous highway pavement maintenance data as input data, use a fully connected layer Dense to extract data features, and use the tanh function for activation.

[0102] S3-3: Set up a FastText feature extraction subnetwork to perform embedding operations on the input highway pavement maintenance text data, converting each character into a 1×20 embedding vector. After the embedding layer, connect a GlobalAveragePooling1D layer to average the embedding vectors, then connect a Dense hidden layer with tanh activation. Finally, use a classifier to obtain the classification result.

[0103] S3-4: Use the feature-based concatenation (Concat) method to perform feature fusion on the output of the sub-network:

[0104] ① Calculate the mean and variance of d neurons:

[0105]

[0106]

[0107] ② The parameters are regularized, and the calculation method is as follows:

[0108]

[0109] ③ A linear transformation is used to shift the data distribution from (0,1) to (-1,1). The calculation method is as follows:

[0110]

[0111] S3-5: Connect a Batch Normalization layer to perform batch normalization of parameters, thereby accelerating model convergence.

[0112] S3-6: Connect a Dense layer as a fusion feature extraction layer.

[0113] S3-7: Connect another Dense layer as the output layer, and use softmax as the loss function of the model. Modify the recognition output results to 7 categories, which correspond to the 7 schemes in the maintenance construction scheme library: "No maintenance", "Crack filling", "Ultra-thin wear layer", "Micro-surfacing", "Milling and repaving the top layer", "Milling and repaving the upper and middle layers", and "Milling and repaving the upper, middle and lower layers".

[0114] S4: Construct a Bayesian multimodal neural network decision model BMNN-EAPMD for highway asphalt pavement maintenance scheme decision-making. The specific content is as follows:

[0115] S4-1: Divide the decision data applied to the highway asphalt pavement maintenance scheme in step S1 into two categories: continuous data and text data (including disease types and pavement structure). Then, encode the text data according to the method in step S1-3 and input it into the decision model.

[0116] S4-2: Set up a Numerical feature extraction subnetwork, take continuous highway pavement maintenance data as input data, use a fully connected layer Dense to extract data features, and use the tanh function for activation.

[0117] S4-3: Set up a FastText feature extraction subnetwork to perform embedding operations on the input highway pavement maintenance text data, so that each character is converted into a 1×20 embedding vector. After the embedding layer, a GlobalAveragePooling1D layer is connected to average the embedding vectors, and then a Dense hidden layer is connected, which is activated by the tanh function. Finally, a classifier is used to obtain the classification result.

[0118] S4-4: Based on MNN-EQPMD, the Tainer and MC Dropout methods are used, specifically: a Dropout layer is inserted after each fully connected layer and before the non-linear activation function.

[0119] S4-5: The outputs of the sub-networks are fused using Concat, with the specific operation method being the same as step S3-4, and the parameters are batch normalized using the Batch Normalization method.

[0120] S4-6: Connect a Dense layer to extract fused features. This Dense layer is also connected using Dropout and then enters the tanh activation function.

[0121] S4-7: Connect another Dense layer as the output layer, and use softmax as the loss function of the model. Modify the recognition output results to 7 categories, which correspond to the 7 schemes in the maintenance construction scheme library: "No maintenance", "Crack filling", "Ultra-thin wear layer", "Micro-surfacing", "Milling and repaving the top layer", "Milling and repaving the upper and middle layers", and "Milling and repaving the upper, middle and lower layers".

[0122] S4-8: Since the output of the model follows a certain probability distribution for each run, statistical analysis of the results yields two output metrics: [Solution, Probability]:

[0123] ①The output of the BMNN-EAPMD model for each run is an x-dimensional vector, where x corresponds to the category output by the model. The index of the maximum value in the model output is taken as the predicted category.

[0124] ② Run the test on the test set n times, and statistically analyze the class prediction results of the n predictions. The statistical structure is as follows: [(M i ,p i ),(M k ,p k )……(M m ,p m )], where M x For the decision category of the BMNN-EAPMD model, p x This represents the probability corresponding to that category.

[0125] ③ The probability of classification in the BMNN-EAPMD model represents the likelihood of a sample belonging to different categories. For an N-class classification problem, the probability of each sample corresponding to each category can also be represented as an N-dimensional vector, with the sum of the probabilities of each category being 1. For the BMNN-EAPMD decision model, if the probability of a sample being classified into class M1 is 0.5 and the probability of being classified into class M2 is 0.4, then the BMNN-EAPMD model cannot accurately determine which category the sample belongs to, indicating that the sample belongs to a marginal case.

[0126] S5: Construct a decision optimization scheme based on the probabilistic uncertainty of maintenance plans. The specific content is as follows:

[0127] S5-1: Regarding the decision outcome [(M i ,p i ),(M k ,p k )……(M m ,p m Sort the data according to probability from largest to smallest, denoted as [(M] x1 ,p x1 ),(M x2 ,p x2 )……(M xn ,p xn )], where p x1 >p x2 >……>p xn , where n is the number of decision schemes given by BMNN-EAPMD, and n≥1;

[0128] S5-2: If there is p x If the confidence level is greater than 0.6, then the scheme is considered to have a high confidence level, and M is selected. x The plan prioritizes the inclusion of corresponding highway sections in the maintenance schedule.

[0129] S5-3: If 0.4≤p x If the ratio is ≤0.6, then the corresponding maintenance plan is considered feasible for the highway section, and a cost-benefit comparison should be conducted. The objective of the plan comparison is to maximize the benefit-cost ratio r, which is calculated as follows:

[0130]

[0131]

[0132] Where C represents maintenance cost, k1, k2, λ1, and λ2 are coefficients adjusted according to actual conditions (default value is 1), B represents maintenance benefit, State represents benefit evaluation index, t represents the decision year, and s represents the number of benefit evaluation indicators used. For the sake of simplification, this paper uses only PCI as the benefit evaluation index when evaluating maintenance benefit B, and the calculation method is as follows:

[0133] B = ΔPCI = PCI t+1 PCI t

[0134] Among them, PCI t As is known, PCI t+1 The PCI value for the following year is obtained from the prediction model. Different predicted PCI values ​​can be obtained by changing the maintenance scheme type for the decision year in the input data. t+1 Based on the prediction results and maintenance costs, a comparison of different options is conducted.

[0135] S5-4: If all solutions have p x A value <0.4 indicates that the decision-making model cannot make an accurate judgment on this highway section based solely on the current database. The decision categories and corresponding probabilities output by the model should be submitted to the manager to assist in decision-making. The manager should conduct an in-depth analysis of the specific pavement defects of the highway, combining this with a comprehensive analysis of funding and benefits, to determine a maintenance plan and correct the results of the decision-making model.

[0136] To verify the effectiveness of the BMNN-EAPMD model, a comparative experiment was designed in this case study. Based on 935 decision data points, the training, validation, and test sets were divided in an 8:1:1 ratio. Comparative experiments were conducted using the MNN-EAPMD, BMNN-EAPMD, Naive Bayes model NB-EAPMD, decision tree model DT-EAPMD, and random forest model RF-EAPMD constructed in this invention. K-fold cross-validation was then performed, with K=10, meaning the dataset was divided into 10 equal parts, and one part was selected as the test set for each experiment, while the remaining nine parts were used as the training set. The classification accuracy of the five models (NB-EAPMD, DT-EAPMD, RF-EAPMD, MNN-EAPMD, and BMNN-EAPMD) on the test set is shown below. Figure 1 As shown in Table 1, the average accuracy rates are as follows. Figure 1As shown in Table 1, the decision accuracies of DT-EAPMD and RF-EAPMD reached 88.9% and 92.0%, respectively; the accuracies of MNN-EAPMD and BMNN-EAPMD were 93.3% and 93.6%, respectively. However, compared with MNN-EAPMD, the BMNN-EAPMD model has a smaller distance between the upper and lower quartiles of its box plot, indicating that the stability of the BMNN-EAPMD model is better than that of MNN-EAPMD. Furthermore, the median accuracy of the BMNN-EAPMD model is higher than that of MNN-EAPMD. In addition, the BMNN-EAPMD model can output additional probabilities as a decision reference.

[0137] Table 1. Average accuracy of each model on the test set.

[0138]

[0139] Furthermore, the confusion matrices of the BMNN-EAPMD model constructed in this case, relative to the seven decision types, are shown in Table 2 for the training set and Table 3 for the test set. M0 to M6 represent "no maintenance", "crack filling", "ultra-thin wear layer", "micro-surfacing", "milling and resurfacing top layer", "milling and resurfacing top and middle layers", and "milling and resurfacing top, middle and bottom layers", respectively. Each row of the confusion matrix represents the true class, and each column represents the predicted class. Specifically, the value in the i-th row and j-th column represents the number of samples in which the model classifies the i-th true class as the j-th class.

[0140] Table 2. Training Set Confusion Matrix

[0141]

[0142]

[0143] Table 3. Confusion Matrix of Test Set

[0144] <![CDATA[M0]]> 29 0 0 0 0 0 0 <![CDATA[M1]]> 0 8 0 0 0 0 0 <![CDATA[M2]]> 0 0 5 0 0 0 0 <![CDATA[M3]]> 0 0 0 8 0 0 0 <![CDATA[M4]]> 0 0 0 0 22 2 0 <![CDATA[M5]]> 0 0 0 0 1 16 1 <![CDATA[M6]]> 0 0 0 0 1 0 3

[0145] To describe the model's decision accuracy for each category, precision and recall are used for comparison. Table 4 shows the precision and recall calculation results of the BMNN-EAPMD model for the seven decision types. Table 4 shows that the class with the lowest precision (85.6%) on the training set is M5, and the class with the lowest recall (89.4%) is M4. On the test set, the class with the lowest precision and recall is M6, both at 75%. Table 4 also shows that M4, M5, and M6 are relatively easy to confuse with the model. They correspond to the three decision schemes of "milling and resurfacing the top layer," "milling and resurfacing the top and middle layers," and "milling and resurfacing the top, middle, and bottom layers," respectively, indicating that the sample features corresponding to these three decision schemes have a certain degree of similarity.

[0146] Table 4. Precision and recall of the BMNN-EAPMD model for seven decision types on the training and test sets.

[0147]

[0148] To demonstrate the effectiveness of the decision optimization scheme constructed in step S5 based on the probabilistic uncertainty of the maintenance scheme, the two northbound lanes of the Xi'an Ring Expressway in Shaanxi Province in 2019 were used as an example to verify and analyze the above decision optimization method. The case involves 81km of continuous single-lane road segments. The estimated unit price of typical maintenance schemes is shown in Table 5, and can be flexibly adjusted according to market conditions.

[0149] Table 5 Typical Maintenance Schemes and Unit Price Estimates

[0150]

[0151] Because the BMNN-EAPMD decision model has a high decision accuracy, for each road segment maintenance unit, the decision scheme with the highest probability given by BMNN-EAPMD is consistent with the actual maintenance scheme in most cases. To reflect the uncertainty of the BMNN-EAPMD decision results and the application of the optimization rules proposed in this paper, Table 6 selects some features of some sections of the northbound second lane of the Xi'an Ring Expressway in Shaanxi Province and the results of model decision and optimization to show them. For example, road segments with the actual maintenance scheme of "no maintenance" are suggested to "mill and repave the upper and middle layers", while road segments that actually adopt "mill and repave the upper, middle and lower layers" are suggested to only "mill and repave the upper and middle layers".

[0152] The optimization of individual road segments constitutes the optimization result for the entire road, with specific costs and benefits shown in Table 7. In the case study, the average PCI of the road surface in 2019 was 82.154, and the average PCI of the road surface in 2020 was 82.625. The actual benefit was 82.625 - 82.154 = 0.471, and the actual maintenance cost was 6.68 million yuan. For the optimized scheme based on the BMNN-EAPMD model, the PK-GRU-EAPPP prediction model can be used to predict the average PCI in 2020. This prediction is then corrected based on the predicted value of the actual maintenance scheme and the actual value in 2020. The result is that if the optimized scheme is adopted, the estimated average PCI in 2020 is 82.656, the estimated benefit is 0.502, and the cost is 5.84 million yuan. Compared to the original scheme, this represents a cost reduction and efficiency improvement.

[0153] Table 6. Characteristics and Maintenance Plans for Some Road Sections

[0154]

[0155] Table 7 Comparison of Actual and Optimized Schemes

[0156]

[0157]

[0158] The technical means disclosed in this invention are not limited to those disclosed in the above embodiments, but also include technical solutions composed of any combination of the above technical features. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications are also considered within the scope of protection of this invention.

Claims

1. A decision-making method for highway asphalt pavement maintenance schemes based on Bayesian multimodal neural networks, characterized in that: Includes the following steps: S1: Construct a highway pavement maintenance information matrix; S2: Select decision indicators for highway pavement maintenance; S3: Construct a multimodal neural network model MNN-EAPMD for decision-making on highway asphalt pavement maintenance schemes; S4: Construct a Bayesian multimodal neural network decision model BMNN-EAPMD for highway asphalt pavement maintenance scheme decision-making; S5: Construct a decision optimization scheme based on the probabilistic uncertainty of maintenance schemes; The specific content of constructing the Bayesian multimodal neural network decision model BMNN-EAPMD for highway asphalt pavement maintenance scheme decision-making in step S4 is as follows: S4-1: Divide the decision data applied to the highway asphalt pavement maintenance scheme in step S1 into two categories: continuous data and text data, and input the text data into the decision model after encoding; S4-2: Set up a Numerical feature extraction subnetwork, take continuous highway pavement maintenance data as input data, use a fully connected layer Dense to extract data features, and use the tanh function for activation; S4-3: Set up a FastText feature extraction subnetwork to perform embedding operations on the input highway pavement maintenance text data, so that each character is converted into a 1×20 embedding vector. After the embedding layer, a GlobalAveragePooling1D layer is connected to average the embedding vectors, and then a Dense hidden layer is connected to activate it with the tanh function. Finally, a classifier is used to obtain the classification result. S4-4: Based on MNN-EQPMD, the MC Dropout method is adopted, specifically: a Dropout layer is inserted after each fully connected layer and before the non-linear activation function; S4-5: The outputs of the sub-networks are fused using Concat, and the parameters are batch normalized using the Batch Normalization method; S4-6: Connect a Dense layer to extract fused features. This Dense layer is also connected using Dropout and then enters the tanh activation function. S4-7: Connect another Dense layer as the output layer, and use softmax as the loss function of the model. Modify the recognition output results to 7 categories, which correspond to the 7 schemes in the maintenance construction scheme library: "No maintenance", "Crack filling", "Ultra-thin wear layer", "Micro-surfacing", "Milling and repaving the top layer", "Milling and repaving the upper and middle layers", and "Milling and repaving the upper, middle and lower layers". S4-8: Since the output of the model follows a certain probability distribution for each run, the statistical analysis of the results can yield two output indicators: [scheme, probability]; The specific content of the statistical method for the output results of each model run in steps S4-8 is as follows: ①The output of the BMNN-EAPMD model in each run is a dimensional vector, For each category corresponding to the model's output, the index of the maximum value in the model's output form is taken as the predicted category. ② Run the test on the test set n times, and statistically analyze the class prediction results of the n predictions. The statistical structure is as follows: ,in For the decision categories of the BMNN-EAPMD model, This represents the probability corresponding to that category. ③ The probability of classification by the BMNN-EAPMD model represents the likelihood of a sample belonging to different categories; for a given sample... In a class classification problem, the probability of each sample corresponding to each class can also be represented as a... A dimensional vector, where the sum of the probabilities of each category is 1; for the BMNN-EAPMD decision model, if a sample is divided into... The probability of class 0.5 is used to classify it. The probability of class is 0.

4. At this time, the BMNN-EAPMD model cannot accurately determine which class the sample belongs to, indicating that the sample belongs to the marginal case. The specific content of constructing the decision optimization scheme based on the probabilistic uncertainty of the maintenance scheme in step S5 is as follows: S5-1: On the decision results Sort according to probability from largest to smallest, denoted as ,in > >……> , where n is the number of decision options given by BMNN-EAPMD, and... ; S5-2: If there is If the confidence level is high, then the proposed solution is considered to have a high degree of confidence and is selected. The plan prioritizes the inclusion of corresponding highway sections in the maintenance schedule. S5-3: If there is If the corresponding maintenance plan is deemed feasible for the highway section, then a cost-benefit comparison should be conducted; the goal of the comparison is to maximize the cost-benefit ratio. The specific calculation method is as follows: in, For maintenance costs, , , , This is a coefficient adjusted according to the actual situation; the default value is 1. For maintenance benefits, As a performance evaluation indicator, For the year of the decision, The number of benefit evaluation indicators used; S5-4: If all solutions have This indicates that the decision-making model cannot make an accurate judgment on the highway section based solely on the current database. It is necessary to submit the decision categories and corresponding probabilities output by the model to the managers to assist in decision-making. The managers should conduct in-depth analysis of the specific road surface defects of the highway, combine it with a comprehensive analysis of funds and benefits, and then determine the maintenance plan to correct the results of the decision-making model.

2. The decision-making method for highway asphalt pavement maintenance scheme based on Bayesian multimodal neural network according to claim 1, characterized in that: The specific method for constructing the highway pavement maintenance information matrix in step S1 is as follows: S1-1: Collect highway pavement maintenance data, including descriptions of highway basic characteristics such as technical grade, pavement structure, maintenance unit, starting and ending station numbers, lanes, year of construction, traffic volume, and meteorological data; and pavement damage index describing highway pavement performance. The following are included: Roadway Quality Index (RQI), Roadway Rutting Index (RDI), Roadway Skid Resistance Index (SRI), Roadway Wear Index (PWI), Roadway Bump Index (PBI), and Structural Strength Index (PSSI); and descriptions of the starting and ending points, names, and lengths of defects in the historical maintenance of highways. S1-2: Extract, transform, and load highway pavement maintenance data from different sources and in different formats to construct an information matrix; S1-3: Preprocessing highway pavement maintenance data; S1-4: Before training each model, randomly select image data to divide it into training and test sets.

3. The decision-making method for highway asphalt pavement maintenance scheme based on Bayesian multimodal neural network according to claim 2, characterized in that: The specific content of constructing the information matrix in step S1-2 is as follows: ① Determine the index data and construct the information table; select 1 kilometer as the length of the road segment unit, that is, the length of the highway segment of the unit sample; set the index data of the data table as "starting point chainage - ending point chainage - up and down lanes"; at the same time, in order to facilitate indexing, convert the "K...+..." of the starting and ending point chainages to integer values; ② Manual data input; For highway pavement maintenance data with small data volume and simple sources, highway pavement maintenance data from different sources are collected into a data table by manual input, including climate data, construction time, road age, and pavement structure; ③ Batch loading; A large amount of highway pavement maintenance data was extracted from various sources and transformed and integrated before being loaded into the database. These sources included periodic inspection reports, construction drawings, detailed damage lists, and traffic volume surveys. For structured data, data reading and format conversion were performed, and the data was matched with index data before being loaded into the data table. For unstructured data, character segmentation was used to clarify the meaning of each part, representing the upstream and downstream directions and the start and end point locations, thereby finding the corresponding index data. At the same time, for descriptions of different damage types and maintenance schemes, character segmentation and fuzzy matching were used to transform them into a unified description, which was then matched with index data before being loaded into the data table. ④ Construct an information matrix; For each road segment unit, the data for each year is used as a feature vector. The road segment unit data are stacked based on spatial coordinates to form a two-dimensional information matrix. Then, considering the time dimension, a time-series-based three-dimensional information matrix is ​​constructed according to the time sequence of the expressway.

4. The decision-making method for highway asphalt pavement maintenance scheme based on Bayesian multimodal neural network according to claim 2, characterized in that: The specific content of highway pavement maintenance data preprocessing in steps S1-3 is as follows: ①One-hot encoding; use one-hot encoding for features with few states; ② Tag Encoding; First, by segmenting, combining, and deduplicating data on pavement structure and defect types of Shaanxi expressways, the following vocabulary was generated: {'AC': 0, 'Medium': 5, 'Fei': 6, 'Fine': 19, 'ATB': 1, 'Coarse': 17, 'SMA': 3, 'Micro-surfacing detachment': 11, 'Cement': 14, 'Insufficient pavement skid resistance': 22, 'Crack': 21, 'Pothole': 9, 'SBS': 2, 'Longitudinal crack': 18, 'Ruts': 23, 'Transverse crack': 13, 'Repair': 7, 'Bursting': 12, 'Pumping': 8, 'Dense transverse cracks': 10, 'Subsidence': 15, 'Network crack': 20, 'Burn':} 16, 'sink':4}, and then map the text data into numerical data according to the vocabulary; if the number of disease types in different sections of the highway is different, resulting in the inability to obtain feature vectors of the same length after conversion, then perform padding operation and add 0s to the end to make the data aligned; ③ Handling missing values; In highway pavement maintenance data, pavement performance data, traffic volume data, and climate data are all continuous variables in time series data. If there are missing values, interpolation can be used to fill them in; For maintenance measure category data, missing values ​​can be treated as a new category. For other empty values, placeholders can be used to fill them; ④ Outlier detection; the interquartile range (IQR) method is used to handle outliers in highway pavement maintenance data; first, the third quartile of the data is calculated. First quartile And the median; then calculate the height of the box. ; Re-determine the range of outliers, with the upper limit being... The lower limit is Finally, data points outside this range are considered outliers and removed. ⑤ Data normalization; To avoid assigning different weights to features of different magnitudes in the model, which would relatively weaken the effect of features with smaller values, min-max normalization is used for scaling: 。 5. The decision-making method for highway asphalt pavement maintenance scheme based on Bayesian multimodal neural network according to claim 1, characterized in that: The specific content of selecting highway pavement maintenance decision indicators in step S2 is as follows: S2-1: Highway asphalt pavement maintenance projects are divided into three types: preventive maintenance, repair maintenance, and routine maintenance. Then, based on the maintenance construction plans recorded in the highway history, statistics are compiled and simplified to classify them. Finally, highway asphalt pavement maintenance plans are divided into six categories: "crack sealing", "ultra-thin wearing course", "micro-surfacing", "milling and repaving of the top layer", "milling and repaving of the upper and middle layers", and "milling and repaving of the upper, middle and lower layers". S2-2: Factors and characteristics influencing maintenance plan decisions; Considering the main factors influencing the decision-making of asphalt pavement maintenance on highways, the following characteristics are selected: distress length, distress type, road age, pavement damage index PCI, mixed equivalent, pavement structure, passenger-to-freight ratio, pavement rutting index RDI, construction time, pavement skid resistance index SRI, PQI, pavement ride quality index RQI, up and down directions, and lanes. S2-3: Input the cleaned highway asphalt pavement maintenance data into the decision tree, classify maintenance plans based on the data of that year, and calculate the impact of each feature on the maintenance plan decision. The importance ranking of features is obtained.

6. The decision-making method for highway asphalt pavement maintenance scheme based on Bayesian multimodal neural network according to claim 2, characterized in that: The specific content of constructing the multimodal neural network model MNN-EAPMD for highway asphalt pavement maintenance scheme decision-making in step S3 is as follows: S3-1: Divide the decision data applied to the highway asphalt pavement maintenance scheme in step S1 into two categories: continuous data and text data. Then, encode the text data according to the method in step S1-3 and input it into the decision model. S3-2: Set up a Numerical feature extraction subnetwork, take continuous highway pavement maintenance data as input data, use a fully connected layer Dense to extract data features, and use the tanh function for activation; S3-3: Set up a FastText feature extraction subnetwork to perform embedding operations on the input highway pavement maintenance text data, so that each character is converted into a 1×20 embedding vector. After the embedding layer, a GlobalAveragePooling1D layer is connected to average the embedding vectors, and then a Dense hidden layer is connected to activate it with the tanh function. Finally, a classifier is used to obtain the classification result. S3-4: Use the feature-based concatenation (Concat) method to perform feature fusion on the output of the sub-network; S3-5: Connect a Batch Normalization layer to perform batch normalization of parameters, thereby accelerating model convergence. S3-6: Connect a Dense layer as a fusion feature extraction layer; S3-7: Connect another Dense layer as the output layer and use softmax as the loss function of the model. Modify the recognition output results to 7 categories, which correspond to the 7 schemes in the maintenance construction scheme library: "No maintenance", "Crack filling", "Ultra-thin wear layer", "Micro-surfacing", "Milling and repaving the top layer", "Milling and repaving the middle layer", and "Milling and repaving the middle and bottom layers".

Images