A road rut evolution prediction method and system based on relative position attention

Abstract

The present application relates to the road maintenance and artificial intelligence technology field, and proposes a kind of road rut evolution prediction method and system based on relative position attention, the method is executed by road rut evolution prediction system, comprising: obtaining target section rut depth time series data and multi-source physical state time series data;Multi-source data is processed, interpolation, alignment, complete and standardization, form rut evolution dataset;Sliding window is used to construct supervised learning sample;The encoder type Transform prediction model of fusion learnable position coding and relative position attention mechanism is constructed, and relative position item is introduced in multi-head self-attention scoring;Model is trained and future multi-step rut depth prediction sequence is output.The present application explicitly introduces relative position information into attention weight calculation process, enhances the modeling capability of rut evolution long-range dependence and phase change, improves multi-step prediction accuracy and stability, facilitates engineering deployment and road maintenance decision.

Description

Technical Field

[0001] This invention relates to the field of intersectional technologies of intelligent operation and maintenance of road engineering and artificial intelligence time series prediction, and more specifically, to a method and system for predicting road rut evolution based on relative position attention. Background Technology

[0002] Rutting is a permanent deformation defect of asphalt pavement caused by the coupled effects of traffic load and environmental factors. It is characterized by long-term accumulation, phased changes and strong nonlinearity, and in severe cases, it can affect driving safety and the service life of the road.

[0003] Existing rut prediction methods mainly include mechanistic models and data-driven models. Mechanistic models rely on material and structural parameters and idealized assumptions, making them difficult to adapt to complex working conditions and multi-source heterogeneous data. Although data-driven models can learn nonlinear relationships, they are prone to problems such as insufficient representation of long-range dependencies and unstable capture of stage-specific mutations in long-sequence and multi-step prediction tasks.

[0004] The Transformer model has a global self-attention mechanism, which is suitable for modeling long sequences. However, if only absolute position encoding is used, it is difficult to explicitly express the physical influence of relative time and relative load interval on rut evolution, thus limiting the prediction accuracy and generalization performance.

[0005] Therefore, there is an urgent need for a prediction method and system that deeply integrates road physical entity data (sensor monitoring, cumulative axle load, rut physical evolution) with improved attention mechanisms to improve the accuracy and engineering usability of long-term rut evolution prediction. Summary of the Invention

[0006] In view of this, the present invention proposes a road rut evolution prediction method and system based on relative position attention, in order to solve the problems of insufficient utilization of relative load interval information and insufficient characterization of stage evolution features in existing rut prediction methods in long-term, multi-step prediction tasks.

[0007] To achieve the above objectives, this invention proposes a road rut evolution prediction method based on relative position attention, comprising: Collect multi-source physical state data of the target road segment; The multi-source physical state data are preprocessed to construct a multivariate rut evolution dataset with axle load as the time axis; Based on the aforementioned multivariate rut evolution dataset, an input sequence matrix and a target sequence of future rut depths are constructed using a sliding window approach. A Transformer model integrating relative position attention is constructed, and the model is trained under supervision using the input sequence matrix and the target sequence of future rut depths. The real-time collected multi-source physical state data is input into the trained model, and the model outputs a sequence of predicted rut depths for future steps.

[0008] Furthermore, the process of collecting multi-source physical state data of the target road segment includes: Obtain rut depth sequences using road surface defect detection equipment; Obtain the cumulative traffic load or cumulative standard axle load sequence through traffic load statistics equipment; Temperature and humidity are obtained through environmental monitoring sensors; Obtain road structure parameters as static or segment-invariant features.

[0009] Furthermore, the preprocessing of the multi-source physical state data includes: Outlier detection and replacement are performed on any feature variable within a sliding window. Outliers are identified using the absolute median difference method and replaced using neighborhood interpolation or median. Variables with different sampling frequencies are mapped onto a unified cumulative standard load grid using interpolation methods, and feature sequences on the unified grid are obtained by linear interpolation or spline interpolation. Grouping by pavement structure type, performing distribution consistency mapping and completion on data from different structure groups; The data after mapping completion is standardized.

[0010] Furthermore, the method for constructing the input sequence matrix and the target sequence of future rut depths using a sliding window approach is as follows: ,

[0011] in Input matrix for historical observations, For future target sequences of rut depth, To achieve a unified axle load mesh in the first t Step multivariate feature vectors, R The depth of the ruts.

[0012] Furthermore, in the multi-head self-attention layer of the Transformer model, a relative position term is introduced to make the attention weights aware of the relative interval. For any attention head, the attention score adopts any of the following forms: or

[0013] in, Rate attention Represents the query and key vector; For the attention dimension, This is a relative distance clipping function. Indicates relative distance.

[0014] Furthermore, during the training process of the Transformer model, a pruning function is introduced to limit the range of relative distances:

[0015] in, This is a relative distance clipping function. Indicates relative distance. K This is the maximum relative distance threshold.

[0016] Furthermore, the loss function during the training of the Transformer model is the mean squared error: .

[0017] Furthermore, the method further includes the following steps: Based on the predicted sequence of future multi-step rut depth, early warning indicators are calculated and early warning levels are output. The early warning information and maintenance suggestions are then written into the intelligent road operation and maintenance platform. The early warning indicators include the predicted maximum rut depth and the predicted rut depth growth rate.

[0018] On the other hand, to achieve the above objectives, this invention proposes a road rut evolution prediction system based on relative position attention, comprising: The data acquisition module is used to connect with rut detection equipment, traffic load statistics equipment and environmental monitoring sensors to collect multi-source physical state time-series data; The data preprocessing module is used to perform anomaly handling, axle load mesh mapping, structural alignment and completion, and standardization on the collected data, and to construct a multivariate evolution dataset of wheel ruts with the cumulative standard axle load as the time axis. The sample construction module is used to construct a sliding window input sequence and a target sequence based on the dataset; The model building and training module is used to build a Transformer model that incorporates relative positional attention and perform supervised training. The prediction output module is used to input real-time data into the model, output a multi-step rut depth prediction sequence, and generate early warning and maintenance decision information.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention employs relative position attention to explicitly incorporate the difference between the axle load action interval and the evolution stage into the attention weight evaluation, enabling the model to simultaneously consider content similarity and relative interval information, thereby improving the accuracy and stability of long sequence multi-step prediction. This invention uses the cumulative standard axis load as the unified evolution time axis, and combines anomaly handling, interpolation, alignment completion and standardization to improve the quality of training data and cross-structure generalization ability; The prediction results obtained using the method of this invention can be directly used for threshold early warning and maintenance decision output, which facilitates the engineering deployment of intelligent road operation and maintenance platform. Attached Figure Description

[0020] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. In the drawings: Figure 1 This is a schematic diagram of the road rut evolution prediction method based on relative position attention of the present invention; Figure 2 This is a schematic diagram of the system structure in an embodiment of the present invention; Figure 3 is a comparison of training curves of different models in the embodiments of the present invention, where (a) is the training curve of the RNN model; (b) is the training curve of the LSTM model; (c) is the training curve of the TCN-GRU model; and (d) is the training curve of the Transformer model that integrates the relative position attention mechanism of the present invention. Figure 4 is a comparison of the rut depth prediction results in the embodiments of the present invention, where (a) is the rut depth prediction result of the TCN-GRU model; and (b) is the rut depth prediction result of the Transformer model that integrates the relative position attention mechanism of the present invention. Detailed Implementation

[0021] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0022] This embodiment proposes a road rut evolution prediction method and system based on relative position attention, such as Figure 1 As shown, road surface defect detection equipment, traffic load statistics equipment, and environmental monitoring sensors are used to collect data on rut depth, cumulative standard axle load cycles, ambient temperature, ambient humidity, and road surface structure parameters of the target road section. Anomaly processing, interpolation, alignment completion, and standardization are performed on multi-source physical state time-series data to form a multivariate rut evolution dataset with the cumulative standard axle load count as the evolution time axis. Construct a sliding window sample, input matrix , and the target sequence ; Construct an encoder-based Transformer model that integrates learnable positional encoding and relative positional attention, and introduce relative distance into the multi-head self-attention scoring. The relevant relative position items are determined, and the effective relative range is limited by a clipping function; The model is trained and outputs a multi-step rut depth prediction sequence, generating early warning and maintenance decision-making information.

[0023] The technical solution of the present invention will be described in detail below. For ease of description, the following symbols are defined: N : Evolution timeline of cumulative traffic load or cumulative standard axle load count (ESAL); R : Rut depth (target variable); T Ambient temperature; H Ambient humidity; : In the unified axle load grid t Step-by-step multivariate feature vector (may contain N,R,T,H and structural parameters, etc. Input sequence matrix; : Target sequence; : Length of the history input window; Future predicted step size; Location index in the attention mechanism; Relative distance; K Maximum relative distance threshold; : Relative distance clipping function; : Query and key vector; Attention dimension; Attention score; Attention weights.

[0024] like Figure 1 As shown, the method of the present invention includes S1 to S6.

[0025] Step S1: Multi-source physical state data acquisition Obtain rut depth sequences using road surface defect detection equipment (such as rut ​​meters, 3D laser / profilometers, etc.). R .

[0026] Obtain the cumulative traffic load or cumulative standard axle load sequence through traffic load statistics equipment (such as weighing systems, traffic radar / axle counting devices, etc.). N .

[0027] Temperature is obtained through environmental monitoring sensors. T With humidity H .

[0028] Optionally, structural parameters such as pavement structure type, surface layer thickness, and base layer thickness can be obtained as static or segment-invariant features.

[0029] Step S2: Data preprocessing and dataset construction with axle load as the time axis For any feature variable of length w Calculate the median within a sliding window and Perform exception handling:

[0030] when If an outlier is detected, it is replaced using median or neighborhood interpolation.

[0031] Map variables at different sampling frequencies to a unified cumulative standard axle load grid. The feature sequence on the unified grid is obtained by using linear interpolation or spline interpolation.

[0032] Group the pavement structures by type, perform distribution consistency mapping on the groups to be completed, and then complete them:

[0033] After mapping, missing points are filled by interpolation on a unified axial load grid, reducing the impact of statistical heterogeneity caused by differences in stiffness of different structures.

[0034] Z-score standardization can be adopted.

[0035] in and Statistically obtained from the training set; or obtained using Min-Max standardization.

[0036] Step S3: Sliding window sample construction Multivariable sequences under a unified grid Constructing sample pairs using a sliding window: ,

[0037] in Input matrix for historical observations, This is a target sequence for future rut depth.

[0038] Step S4: Constructing the Relative Position Attention Transformer Model Input matrix An embedding representation is obtained through linear mapping, and then a learnable positional encoding is superimposed. P To enhance the adaptive expression of semantics at the evolutionary stage.

[0039] In a multi-head self-attention layer, a relative position term is introduced to make the attention weights aware of the relative interval. For any attentional head, score the attention level. It may be in any of the following forms: or

[0040] The clipping function is:

[0041] Attention weights are determined by softmax get:

[0042] Attention output is obtained by weighting and converging value vectors based on attention weights, and comprehensive modeling of long-range dependencies and local stage changes is achieved by stacking encoder layers.

[0043] Step S5: Model Training Supervised learning is used to train the model parameters, and the mean squared error can be used as the loss function.

[0044] Gradient descent-type optimization algorithms can be used to update model parameters until convergence.

[0045] Step S6: Predictive Output and Early Warning / Maintenance Decision Generation The latest multi-source physical state data obtained from monitoring is processed according to S2 and S3 to obtain the input matrix, which is then input into the trained model to output a prediction sequence of future multi-step rut depths. .

[0046] based on Calculate early warning indicators and output early warning levels, such as predicting the maximum rut depth. and the predicted growth rate of rut depth .

[0047] The early warning information and maintenance suggestions (such as preventive milling, overlay, and load limit suggestions) can be further incorporated into the road intelligent operation and maintenance platform to achieve engineering closed-loop application.

[0048] like Figure 2 As shown, the system of the present invention includes: Data acquisition module: interfaces with rut detection equipment, traffic load statistics equipment, and environmental monitoring sensors for data collection. R,N,T,H and structural parameters; Data preprocessing module: used to perform anomaly handling, shaft load mesh mapping interpolation, structural alignment completion and standardization; Sample building module: used for constructing samples using a sliding window. ,

[0049] Model building and training module: used to build and train a Transformer model that integrates learnable positional encoding and relative positional attention; Prediction output module: Used to output the prediction sequence of future multi-step ruts and generate early warning and maintenance decision information.

[0050] Road rutting evolution prediction based on full-scale loop data This embodiment uses a long-term service monitoring scenario of a road as an example to illustrate the dataset construction process of the method of the present invention. The target road section is equipped with a pavement defect detection device, a traffic load statistics device, and an environmental monitoring device to periodically acquire rut depth data, cumulative standard axle load count data, ambient temperature data, and ambient humidity data; at the same time, it acquires structural parameters such as pavement structure type, surface layer thickness, and base layer thickness.

[0051] In this embodiment, the data acquisition cycle can be updated daily, weekly, or according to a preset axle load increment. Due to the inconsistent sampling frequencies of different devices, the raw data may contain missing measurements, abnormal fluctuations, and inconsistent timelines.

[0052] For the collected multi-source time-series data, the absolute median difference method is first used for anomaly identification. For any feature sequence of length... w Calculate the median within a sliding window And absolute median:

[0053] when When this occurs, the observation is identified as an outlier and replaced with an interpolated value from a neighboring valid observation.

[0054] Then, based on the cumulative standard axle load cycles As a unified evolution time axis, the rut depth, temperature, humidity and structural features under different sampling frequencies are mapped onto a unified axle load grid, and linear interpolation or spline interpolation is used to obtain the feature values ​​on the unified grid.

[0055] For structural data groups with significant gaps or sparse sampling, alignment and completion are performed by grouping them according to pavement structure type. In one implementation, distribution consistency mapping is performed using statistics from a reference structural group and the structural group to be completed.

[0056] After mapping, interpolation is performed on a unified axis load grid to complete the mapping.

[0057] The completed data is standardized to obtain a multivariate evolution dataset of ruts, which provides an input basis for subsequent sample construction and model training.

[0058] Supervised learning samples are constructed based on the obtained multivariate evolution dataset of wheel ruts. Let the multivariate sequence on the uniform axle load grid be... Each The input sequence matrix is ​​constructed using a sliding window method. With the target sequence : ,

[0059] The input sequence matrix is ​​linearly mapped to obtain an embedding representation, which is then superimposed with a learnable positional encoding; this is subsequently input into an encoder-based Transformer model. For any attention head, at position... i Position j Attention scores can be given in any of the following formats: or

[0060] in, , .

[0061] By using the aforementioned relative position term, the model can simultaneously consider the similarity of characteristic values ​​and the relative axle load interval when calculating the correlation between current and historical observations, thereby enhancing its ability to model the cumulative deformation patterns and stage-by-stage changes of ruts.

[0062] The mean squared error loss function is used during the training phase:

[0063] The model parameters are updated using gradient descent-type optimization algorithms until the loss converges or a preset stopping condition is met.

[0064] In the application phase, the newly acquired multi-source physical state data of the target road segment is preprocessed and sample constructed before being input into the trained model, which outputs a multi-step rut depth prediction sequence. .based on and rut growth rate The system generates early warning levels and outputs maintenance recommendations to the road maintenance terminal.

[0065] This embodiment demonstrates that the present invention can achieve stable prediction of the road rutting process and has good engineering applicability.

[0066] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for predicting road rut evolution based on relative position attention, characterized in that, include: Collect multi-source physical state data of the target road segment; The multi-source physical state data are preprocessed to construct a multivariate rut evolution dataset with axle load as the time axis; Based on the aforementioned multivariate rut evolution dataset, an input sequence matrix and a target sequence of future rut depths are constructed using a sliding window approach. A Transformer model integrating relative position attention is constructed, and the model is trained under supervision using the input sequence matrix and the target sequence of future rut depths. The real-time collected multi-source physical state data is input into the trained model, and the model outputs a sequence of predicted rut depths for future steps.

2. The method according to claim 1, characterized in that, The process of collecting multi-source physical state data of the target road segment includes: Obtain rut depth sequences using road surface defect detection equipment; Obtain the cumulative traffic load or cumulative standard axle load sequence through traffic load statistics equipment; Temperature and humidity are obtained through environmental monitoring sensors; Obtain road structure parameters as static or segment-invariant features.

3. The method according to claim 1, characterized in that, The preprocessing process for the multi-source physical state data includes: Outlier detection and replacement are performed on any feature variable within a sliding window. Outliers are identified using the absolute median difference method and replaced using neighborhood interpolation or median. Variables with different sampling frequencies are mapped onto a unified cumulative standard load grid using interpolation methods, and feature sequences on the unified grid are obtained by linear interpolation or spline interpolation. Grouping by pavement structure type, performing distribution consistency mapping and completion on data from different structure groups; The data after mapping completion is standardized.

4. The method according to claim 1, characterized in that, The method for constructing the input sequence matrix and the target sequence of future rut depths using a sliding window approach is shown below: ， ， in Input matrix for historical observations, For future target sequences of rut depth, To achieve a unified axle load mesh in the first t Step multivariate feature vectors, R The depth of the ruts.

5. The method according to claim 1, characterized in that, In the multi-head self-attention layer of the Transformer model, a relative position term is introduced to make the attention weights aware of the relative interval. For any attention head, the attention score takes any of the following forms: or , in, Rate attention Represents the query and key vector; For the attention dimension, This is a relative distance clipping function. Indicates relative distance.

6. The method according to claim 1, characterized in that, During the training process of the Transformer model, a clipping function is introduced to limit the range of relative distances: ， in, This is a relative distance clipping function. Indicates relative distance. K This is the maximum relative distance threshold.

7. The method according to claim 1, characterized in that, The loss function used in the training process of the Transformer model is the mean squared error: 。 8. The method according to claim 1, characterized in that, The method further includes the following steps: Based on the predicted sequence of future multi-step rut depth, early warning indicators are calculated and early warning levels are output. The early warning information and maintenance suggestions are then written into the intelligent road operation and maintenance platform. The early warning indicators include the predicted maximum rut depth and the predicted rut depth growth rate.

9. A road rut evolution prediction system based on relative position attention, used to implement the method as described in any one of claims 1 to 8, characterized in that, include: The data acquisition module is used to connect with rut detection equipment, traffic load statistics equipment and environmental monitoring sensors to collect multi-source physical state time-series data; The data preprocessing module is used to perform anomaly handling, axle load mesh mapping, structural alignment and completion, and standardization on the collected data, and to construct a multivariate evolution dataset of wheel ruts with the cumulative standard axle load as the time axis. The sample construction module is used to construct a sliding window input sequence and a target sequence based on the dataset; The model building and training module is used to build a Transformer model that incorporates relative positional attention and perform supervised training. The prediction output module is used to input real-time data into the model, output a multi-step rut depth prediction sequence, and generate early warning and maintenance decision information.

Images