Traffic speed prediction method and device based on self-attention neural control

Abstract

The embodiment of the application discloses a traffic speed prediction method and device based on self-attention neural control, relates to the technical field of intelligent transportation, and can improve the precision of traffic speed prediction while maintaining low calculation complexity. The application comprises: obtaining historical data of traffic speed, and generating an original input sequence; establishing an AB-NCDE model, and performing a forward propagation process on the original input sequence; establishing a loss function and performing a backward propagation process; and using the trained AB-NCDE model to predict traffic speed. The scheme is used for traffic speed prediction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent transportation, and in particular to a traffic speed prediction method and device based on self-attention neural control. BACKGROUND

[0002] In the current various intelligent traffic control systems, how to accurately predict the real-time road traffic speed has always been a difficult problem. Many current analysis algorithms are mainly derived from the fields of fluid mechanics, statistics, and logistics, and model and analyze roads and traffic nodes as a material transportation channel. These methods can achieve traffic speed prediction to some extent, but their accuracy is difficult to improve. The main reason is that each road and traffic node has certain chaotic system characteristics in actual application, and existing solutions are difficult to accurately handle the spatiotemporal dependence relationship. For example:

[0003] Traditional time series analysis methods, such as ARIMA, SARIMA, etc., are used to predict traffic speed. These methods are based on the stationarity and linearity of data. These methods are sufficient for stable fluid transport systems and logistics sorting systems, but they have limited prediction effect when dealing with complex and nonlinear traffic speed data.

[0004] Machine learning methods, such as support vector machines (SVM) and random forests, are used to predict traffic speed. These methods have better versatility and accuracy than time series analysis methods, but they require a lot of feature engineering before implementation. They are suitable for short-term prediction scenarios, such as obtaining speed-related parameters in traffic tool scheduling optimization, such as data flow, vehicles, ships, and aircraft. However, they have limited ability to handle long time series data.

[0005] Deep learning methods, such as LSTM and GRU recurrent neural networks, are used to predict traffic speed. These methods can handle time series data and are currently the most commonly used method in traffic management. However, as China's transportation network becomes more complex and larger, these methods have limited ability to model the spatiotemporal dependence relationship of complex transportation networks, and there is little room for further improvement.

[0006] In summary, current technical solutions have problems such as insufficient spatiotemporal dependence modeling and high computational complexity. Therefore, how to improve the accuracy of traffic speed prediction without significantly increasing computational complexity has become a key research topic. SUMMARY

[0007] The embodiments of the present invention provide a traffic speed prediction method and apparatus based on self-attention neural control, which can improve the accuracy of traffic speed prediction while maintaining low computational complexity.

[0008] To achieve the above objectives, the embodiments of the present invention adopt the following technical solutions:

[0009] Firstly, the methods provided by embodiments of the present invention, such as Figure 3 As shown, it includes:

[0010] S1. Obtain historical traffic speed data and generate the original input sequence;

[0011] S2. Establish the AB-NCDE model and perform a forward propagation process on the original input sequence;

[0012] S3. Establish the loss function and execute the backpropagation process;

[0013] S4. Use the trained AB-NCDE model to predict traffic speed.

[0014] Specifically, S1 includes: obtaining vehicle speed data for each road segment recorded as a JSON string from the map data server, identifying road segments using the linkid identifier, and extracting the average vehicle speed for each road segment; the discrete original input sequence is represented as: , This represents the n observed velocity values. These represent the n time points corresponding to the observed velocity values.

[0015] The forward propagation process on the original input sequence includes: Continuous time series values ​​are generated using function interpolation. Using the timing control function f, for To latent variables Mapping to obtain latent variables Evolving Relationships ; By using the graph convolution control function g, the latent variables are... To new latent variables Perform mapping and compute new latent variables. Evolving Relationships ;pass The output layer obtains the predicted velocity sequence. .

[0016] In the AB-NCDE model, the NCDE component includes: , This is the initial state of the model. It is the state of the model at time T. These are the parameters of the neural network. The integrand is denoted as t∈[0,T]; the temporal control component of the AB-NCDE model includes: extracting temporal features. The method is as follows: ;Will The input is a self-attention module employing a multi-head attention mechanism. After the attention matrix is ​​calculated, it is normalized to obtain the processed temporal features. , For the processed time series features After feature extraction through a linear layer, the data is normalized and then passed through the activation function tanh to obtain the final temporal output z. .

[0017] Furthermore, the latent variables are obtained through the first differential solver. Evolving Relationships The first differential solver includes: This represents the initial state of the hidden variable H. This represents the state of the latent variable H at time T; new latent variables are obtained through the second differential solver. Evolving Relationships The second differential solver includes: ,in, This represents the initial state of the hidden variable Z. This represents the state of the latent variable Z at time T. Latent variable The predicted velocity sequence is given through an output layer. .

[0018] The graph convolution control mechanism of the AB-NCDE model includes: ,in, The input to the graph convolution control function g is equal to the latent variable in this invention. The instantaneous value, This indicates that features were extracted after layer normalization;

[0019] for A linear transformation is performed to obtain the query value and key value. Then, the attention score between them is calculated, and a dynamic adjacency matrix is ​​generated using softmax. Perform graph convolution and layer normalization to obtain intermediate parameters. ,in, , indicating the highest Chebyshev order. Indicates trainable weights. Represents the Chebyshev polynomial. This represents the renormalized Laplace matrix. , It is the normalized Laplace matrix The largest eigenvalue of , where I is the identity matrix. D is the degree matrix; according to The output result x of the AB-NCDE model is obtained. .

[0020] Secondly, the apparatus provided in the embodiments of the present invention includes:

[0021] The preprocessing module is used to acquire historical traffic speed data and generate the raw input sequence;

[0022] The first computation module is used to run the AB-NCDE model and perform a forward propagation process on the original input sequence;

[0023] The second calculation module is used to establish the loss function and execute the backpropagation process;

[0024] The execution module is used to predict traffic speed using the trained AB-NCDE model.

[0025] The preprocessing module is specifically used to obtain vehicle speed data for each road segment recorded in JSON string format from the map data server, identify road segments using the linkid identifier, and extract the average vehicle speed for each road segment; the discrete original input sequence is represented as: ,in, This represents the observed speed values ​​of n vehicles. This represents the n time points corresponding to the observation.

[0026] The first calculation module is used for... Continuous time series values ​​are generated using function interpolation. Using the timing control function f, for To latent variables Mapping to obtain latent variables Evolving Relationships ; By using the graph convolution control function g, the latent variables are... To new latent variables Perform mapping and compute new latent variables. Evolving Relationships ;pass The output layer obtains the predicted velocity sequence. .

[0027] The traffic speed prediction method and device based on self-attention neural control provided by the embodiments of the present application deeply integrate neural controlled differential equations (NCDE) and a self-attention mechanism, construct a new paradigm of spatiotemporal joint modeling, realize dynamic fusion of spatiotemporal features by designing the interaction of the time series control function f and the graph convolution control function through the hidden variables H and Z, realize traffic speed prediction based on the self-attention neural controlled differential equation (AB-NCDE, Attention-Based Neural Controlled Differential Equations), and thus more effectively model the spatiotemporal dependence of traffic speed and improve the prediction accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0029] Figure 1 The flow of input and output provided by the embodiments of the present application;

[0030] Figure 2 The overall framework diagram of model calculation provided by the embodiments of the present application;

[0031] Figure 3 The method flowchart provided by the embodiments of the present application. DETAILED DESCRIPTION

[0032] For those skilled in the art to better understand the technical solutions of the present application, the present application will be further described in detail below in combination with the drawings and specific embodiments. Hereinafter, the embodiments of the present application will be described in detail, and the examples of the embodiments are shown in the drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and are only used to explain the present application, and cannot be explained as a limitation of the present application. Those skilled in the art can understand that, unless specifically stated, the singular forms "a", "an" and "the" used herein can also include the plural forms. It should be further understood that the phrase "comprising" used in the specification of the present application means that the features, integers, steps, operations, elements and / or components exist, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we say an element is "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, or there can be an intermediate element. In addition, "connected" or "coupled" used herein can include wireless connection or coupling. The phrase "and / or" used herein includes any one of the associated listed items and all combinations of the associated listed items. Those skilled in the art can understand that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as that of ordinary skilled in the art to which the present application belongs. It should also be understood that terms such as those defined in a general dictionary should be understood to have meanings consistent with those in the context of the prior art, and unless defined as such, should not be interpreted in an idealized or overly formal sense.

[0033] The present embodiment designs a speed prediction scheme based on self-attention neural controlled differential equation, first explains the principle of AB-NCDE (Attention-Based Neural Controlled Differential Equations) based on self-attention neural controlled differential equation.

[0034] Neural Controlled Differential Equations (NCDEs) is a continuous-time dynamic model that combines the concepts of neural networks and Controlled Differential Equations (CDEs). NCDEs parameterize the control function in the controlled differential equations through a neural network, enabling the model to capture complex, continuous-time dynamic behavior. This approach is particularly suitable for handling time series data, effectively modeling the continuous changes in data over time without relying on traditional feature extraction methods. By designing the control signal and differential equation, NCDEs provide a certain degree of interpretability while maintaining high flexibility and adaptability, efficiently processing large-scale time series data.

[0035] The form of Neural Controlled Differential Equations is as follows:

[0036]

[0037] where is the initial state of the system, is the state of the system at time T, is the parameter of the neural network. In the method proposed in this embodiment, the time series value is generated by discrete raw input through spline interpolation, which is a preliminary fitting of the data generation process. Since is twice continuously differentiable with respect to time, the form of NCDE can be further rewritten as:

[0038]

[0039] The specific implementation of the time series control function f. For input , first pass through a linear transformation layer, then pass through the activation function ELU, and then pass through layer normalization, which is to extract the time series features , the calculation method is as follows:

[0040]

[0041] For time series features , enter a self-attention module that adopts a multi-head attention mechanism. After the attention matrix is calculated, pass through a layer normalization again to get the processed time series features :

[0042]

[0043] For processed time series features , through a linear layer to extract features, and then through layer normalization, and finally through the activation function tanh, the output result z of the final timing control function f is obtained:

[0044]

[0045] The specific implementation of the graph convolution control function g is introduced below. For the input , also first through a linear transformation layer, then through the activation function ELU, and then through layer normalization to extract features , the calculation method is as follows:

[0046]

[0047] Then take a similar self-attention method to get the dynamic adjacency matrix . For the feature , the query value Q and the key value K' are obtained through linear transformation, and the attention score between Q and K' is calculated, and the dynamic adjacency matrix is generated through softmax. In order to ensure the symmetry of , the values of Q and K are the same.

[0048] After obtaining the adjacency matrix , the normalized Laplacian matrix is calculated, and the calculation method is as follows, where D is the degree matrix and I is the identity matrix:

[0049]

[0050] Since the subsequent calculation adopts Chebyshev polynomial approximation, it is necessary to re-normalize the Laplacian matrix here, so that all eigenvalues are between , the calculation method is as follows, where is the re-normalized Laplacian matrix, is the maximum eigenvalue of , and I is the identity matrix:

[0051]

[0052] Subsequently, graph convolution and layer normalization are performed to obtain :

[0053]

[0054] where K represents the highest Chebyshev order, which is a variable hyperparameter, is a trainable weight, is a Chebyshev polynomial.

[0055] For the processed feature , the features are extracted through a linear layer, then normalized by layer normalization, and finally passed through the activation function tanh to obtain the output result x of the graph convolution control function g:

[0056]

[0057] The overall processing flow of the AB-NCDE (Attention-Based Neural Controlled Differential Equations) model is shown in Figure 1 . The overall flow of the model adopts the idea of Neural Controlled Differential Equations, and the control function used internally for time domain processing and graph convolution adopts the mechanism of Self-Attention. By combining the two, a variable adjacency matrix and temporal dependence relationship can be constructed from the input sequence itself without relying on external information, and a reliable prediction result can be given.

[0058] Specifically, the discrete original input sequence is obtained by observing the vehicle speed, where represents the observed n speed values, represents the corresponding n time points at the time of observation. First, the function interpolation method is used to generate the continuous time sequence value that can be used for integration ; then the time control function f is used to map to the hidden variable , and the real-time evolution relationship of the hidden variable is calculated ; then according to the graph convolution control function g, the hidden variable is mapped to the new hidden variable , and the real-time evolution relationship of the hidden variable is calculated ; finally the prediction sequence of the speed is given through an output layer. After the control functions f and g are determined, and can be solved by the corresponding differential equation numerical solver, and the calculation method is as follows:

[0059]

[0060]

[0061] The hidden variable is mapped to the prediction sequence of the speed through an output layer.

[0062] Overall, the dual control function architecture is improved in this embodiment, and the logical sequence is as follows: the timing control function f: the multi-head self-attention mechanism based on the Transformer extracts the timing dependence feature through the process of linear transformation → ELU activation → layer normalization → multi-head attention → residual connection. And the graph convolution control function g is also improved, and the logical sequence is as follows: constructing a dynamic adjacency matrix → calculating a normalized Laplacian matrix → Chebyshev polynomial approximation of graph convolution operation to capture spatial dependence. Then an adaptive feature fusion mechanism is adopted, which deeply couples the timing features and spatial features in the hidden space through feature splicing and weighted fusion strategy, forming a representation vector containing spatio-temporal joint information. Overall, the NCDE and self-attention mechanism are deeply fused for the first time in this embodiment, and a new paradigm for spatio-temporal joint modeling is constructed. Traditional methods mostly use spatio-temporal separation modeling or simple splicing strategy, and this scheme realizes the dynamic fusion of spatio-temporal features through the interactive design of timing control function and graph convolution control function.

[0063] In combination with specific application scenarios, the overall framework of the model established in this embodiment is as shown in Figure 2 The specific implementation is as follows:

[0064] Step one: Obtain the speed data of the Internet map provider, the time range is from 00:00 on January 1, 2025 to 08:58 on February 28, 2025. In the preferred scheme, the time span is about two months, and it is recorded once every two minutes. It is recorded in the form of a json string, as follows: {"linkid":"+gxcyTl / ejZCJArEZGPFnA==","speed":58.08000183105469,"state":1},{"linkid":"XWSYoHrsTilpNCBkXyyhkQ==","speed":28.600000381469727,"state":1},{"linkid":"Hzby+nt7tW / XuZg / BMIl8A==","speed":28.600000381469727,"state":1}.

[0065] linkid is the unique identifier of the link; speed represents the average speed of the link. After data preprocessing, a discrete original input sequence is formed, saved as a numpy file, and the shape is 300*1680. The training set, validation set, and test set are divided in the ratio of 6:2:2.

[0066] Step 2: Based on the principle of AB-NCDE, the input sequence at 10 time points is sequentially interpolated using function interpolation to generate continuous time series values ​​suitable for integration. Then, using the timing control function f, for... To latent variables Perform mapping and calculate latent variables. Evolving Relationships Then, based on the graph convolution control function g, the latent variables are... To new latent variables Perform mapping and calculate latent variables. Evolving Relationships ;at last The predicted velocity sequence is given through an output layer. (10 per group). This step completes one forward propagation process. The multi-head attention mechanism is set to 2 heads, and the Chebyshev order of the graph convolution is also set to 2.

[0067] Step 3: Calculate the loss function (the loss function is to minimize the MAE error between the predicted value and the true value), backpropagate, and update the parameters. Steps 2 and 3 together constitute one round of training.

[0068] Step 4: Repeat steps 2 and 3, i.e., train the input data for 50 rounds. If the prediction performance on the validation set does not improve after 10 consecutive rounds, terminate the training prematurely.

[0069] Step 5: Perform velocity prediction on the test set and calculate the mean absolute error (MAE) and root mean square error (RMSE) as prediction metrics.

[0070] Table 1 shows the MAE and RMSE errors under different prediction windows in this embodiment:

[0071] Table 1

[0072] timestep MAE RMSE 6 min 3.51 5.32 12 min 3.71 5.80 18 min 3.83 6.15 24 min 3.95 6.42 30 min 4.06 6.66 36 min 4.12 6.83 42 min 4.22 7.00 48 min 4.35 7.14 54 min 4.38 7.25 60 min 4.47 7.38 average 4.06 6.63

[0073] Some existing solutions suffer from strong data dependencies. For example, current methods have high requirements for data quality and completeness, but limited ability to handle missing or outlier data. Other existing solutions lack sufficient modeling of spatiotemporal dependencies. For instance, traffic speed is influenced by various factors such as road network structure and traffic flow, and current methods struggle to effectively model these complex spatiotemporal dependencies. Furthermore, existing technologies also suffer from high computational complexity. For example, as the scale of the traffic network and the amount of data increase, the computational complexity of existing methods increases significantly, making it difficult to meet the needs of real-time prediction.

[0074] The main reason for the above problems is that the existing scheme mainly adopts space-time separation modeling or a simple splicing strategy, and the traffic speed prediction method and device based on self-attention neural control provided in the embodiment of the application deeply integrates neural controlled differential equations (NCDE) and a self-attention mechanism, constructs a new paradigm of space-time joint modeling, realizes dynamic fusion of space-time features by designing the interaction of the time sequence control function f and the graph convolution control function through the hidden variables H and Z, realizes traffic speed prediction based on the self-attention neural controlled differential equation (AB-NCDE, Attention-Based Neural Controlled Differential Equations), and thus more effectively models the space-time dependence relationship of traffic speed and improves the prediction accuracy.

[0075] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each of the embodiments mainly describes the difference from other embodiments. Especially, the device embodiment is described simply because it is basically similar to the method embodiment. The related parts can be referred to the part of the method embodiment. The above is merely a specific implementation of the application, but the protection scope of the application is not limited to this. Any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the application, which should be covered in the protection scope of the application. Therefore, the protection scope of the application should be subject to the protection scope of the claims.

Claims

1. A neural controlled traffic speed prediction method based on self-attention, characterized in that, The method comprises the following steps: S1, obtaining historical traffic speed data and generating an original input sequence; S2, establishing an AB-NCDE model and performing a forward propagation process on the original input sequence; S3, establishing a loss function and performing a backward propagation process; S4, using the trained AB-NCDE model to predict traffic speed; In S1, the method comprises the following steps: Obtaining vehicle speed data corresponding to each road section recorded in the form of a json string from a map data server, identifying the road section through a linkid identifier, and extracting the average vehicle speed corresponding to each road section; The discrete raw input sequence is represented as: wherein, denotes the n observed velocity values, denotes the n time points corresponding to the n velocity values, respectively. The forward propagation process on the original input sequence comprises the following steps: To Generating continuous time series values by function interpolation ; Using the time control function f, the time control function f is used to Map the hidden variable to the hidden variable The evolution relationship of the hidden variable ; By graph convolution control function g, the hidden variable is mapped to a new hidden variable , the real-time evolution relationship of the new hidden variable is calculated ; By the output layer to obtain a predicted sequence of velocities ; In the AB-NCDE model, the NCDE part includes: , is the initial state of the model z, is the state of the model z at time T, is the parameter of the neural network, the function form in the bracket carries a semicolon represents the integrand function, is the state of the model z at time t, t∈[0, T]; The timing control link of the AB-NCDE model comprises extracting timing features in the following manner: ; Will The input is input into a self-attention module adopting a multi-head attention mechanism, and after attention matrix calculation, normalization processing is performed to obtain processed time sequence features , ; For the processed timing features , after extracting features through a linear layer, normalization processing is performed, and then an activation function tanh is passed to obtain the final timing output z, ; acquiring the hidden variable by a first differential solver of the instantaneous evolution relationship comprising in the first differential solver: representing an initial state of the hidden variable H, representing a state of the hidden variable H at a T time, representing a state of the hidden variable H at a t time; acquiring a new latent variable by a second differential solver of the instantaneous evolution relationship , in which the second differential solver comprises: , wherein, represents an initial state of the latent variable Z, represents a state of the latent variable Z at the T moment, and the latent variable obtaining a predicted sequence of the speed through an output layer .

2. The method of claim 1, wherein, The graph convolution control link of the AB-NCDE model comprises the following steps: wherein, denotes the input to the graph convolution control function g, denotes the layer normalized extracted features; For Linear transformation is performed to obtain query values and key values, and then attention scores between them are calculated to generate a dynamic adjacency matrix through softmax Graph convolution and layer normalization are performed to obtain intermediate parameters , wherein , represents the highest Chebyshev order, represents a trainable weight, represents a Chebyshev polynomial, represents a renormalized Laplacian matrix, , is the maximum eigenvalue of the normalized Laplacian matrix , I is a unit matrix, , D is a degree matrix; According to obtaining an output result x of the graph convolution control function g, .

3. A device for neural controlled traffic speed prediction based on self-attention, characterized in that, The method comprises the following steps: A preprocessing module is configured to obtain historical traffic speed data and generate an original input sequence; A first calculation module is configured to run an AB-NCDE model and perform a forward propagation process on the original input sequence; A second calculation module is configured to establish a loss function and perform a backward propagation process; An execution module is configured to use the trained AB-NCDE model to predict traffic speed; The preprocessing module is specifically configured to Obtain vehicle speed data corresponding to each road section recorded in the form of a json string from a map data server, identify the road section through a linkid identifier, and extract the average vehicle speed corresponding to each road section; The discrete raw input sequence is represented as: wherein, denotes the n observed velocity values, denotes the n time points corresponding to the n velocity values, respectively. The forward propagation process on the original input sequence comprises the following steps: To Generating continuous time series values by function interpolation ; Using the time control function f, the time control function f is used to Map the hidden variable Map, get the hidden variable The evolution relationship of the hidden variable ; The hidden variable is mapped to a new hidden variable by a graph convolution controlled function g The new hidden variable is computed By the output layer to obtain a predicted sequence of velocities ; In the AB-NCDE model, the NCDE part includes: , is the initial state of the model z, is the state of the model z at time T, is the parameter of the neural network, the function form in the bracket carries a semicolon represents the integrand function, is the state of the model z at time t, t∈[0, T]; The timing control link of the AB-NCDE model comprises extracting timing features in the following manner: ; Will The input is input into a self-attention module adopting a multi-head attention mechanism, and after attention matrix calculation, normalization processing is performed to obtain processed time sequence features , ; For the processed timing features , after extracting features through a linear layer, normalization processing is performed, and then an activation function tanh is passed to obtain the final timing output z, ; acquiring the hidden variable by a first differential solver of the instantaneous evolution relationship comprising in the first differential solver: an initial state of the hidden variable H, a state of the hidden variable H at time T, a state of the hidden variable H at time t; obtaining a new latent variable by a second differential solver of the instantaneous evolution relationship , in which the second differential solver comprises: , wherein, represents an initial state of the latent variable Z, represents a state of the latent variable Z at the T moment, and the latent variable obtaining a predicted sequence of the velocity through an output layer .

Images