Traffic prediction method and system based on self-supervised learning

Abstract

This invention provides a traffic prediction method and system based on self-supervised learning. The method includes the following steps: acquiring traffic flow data; preprocessing traffic flow data; constructing a traffic prediction model based on a neural network model; capturing the spatiotemporal dependencies of traffic flow data to obtain the original spatiotemporal embedding; performing data masking on the traffic flow training data to obtain enhanced traffic flow data; capturing the spatiotemporal dependencies of the enhanced traffic flow data to obtain the enhanced spatiotemporal embedding; then obtaining spatial heterogeneity loss functions and temporal heterogeneity loss functions respectively; performing traffic flow prediction using a multilayer perceptron to obtain the prediction loss function; finally calculating the total model loss function; optimizing the model parameters of the traffic prediction model based on the total model loss function; inputting traffic flow test data into the traffic prediction model to obtain the traffic prediction results for the target city. This invention has the effect of accurately predicting urban traffic conditions.

Description

Technical Field

[0001] This invention belongs to the field of urban planning and management technology, specifically relating to a traffic prediction method and system based on self-supervised learning. Background Technology

[0002] The importance of traffic forecasting in urban planning and management is self-evident. Accurate traffic forecasting enables effective traffic management, reduces congestion, and enhances the sustainability of urban transportation systems. Among these, grid-based traffic flow forecasting plays a crucial role in understanding and managing urban mobility dynamics. By dividing the city into manageable grids and forecasting traffic flow within each grid, city planners can conduct detailed analyses of traffic patterns, facilitating targeted interventions and infrastructure planning. Over the years, traffic forecasting methods have evolved through three phases: traditional statistical models, machine learning techniques, and deep learning methods. Each phase represents a leap in forecast accuracy and the ability to handle complex spatiotemporal data. The adoption of deep learning in traffic forecasting marks a significant milestone, enabling unprecedented accuracy by capturing complex patterns in traffic flow using large datasets. This evolution underscores the increasing complexity of urban transportation systems and the urgent need for advanced forecasting models to address the challenges of the modern urban environment.

[0003] Traffic data contains spatial and temporal heterogeneity, reflecting the variability of traffic flow across different regions and time steps. These variations are crucial for understanding the dynamics of urban traffic and require sophisticated predictive models to capture this dynamism. The spatial and temporal heterogeneity in traffic data poses a challenge to traditional predictive methods, but also offers opportunities to incorporate heterogeneity into the modeling process to improve predictive accuracy. Therefore, recognizing and modeling the spatial and temporal heterogeneity in traffic data is essential for developing accurate and reliable traffic prediction models.

[0004] Current traffic prediction models have some limitations in capturing spatiotemporal heterogeneity. Some models attempt to incorporate temporal features (such as periodicity and holidays) to capture temporal heterogeneity, but these are predefined features and may not fully capture the complexity of urban traffic patterns. Predefined spatial heterogeneity features are often obtained using graph embedding based on adjacency matrices, ignoring the complexity and diversity of regions. Reliance on handcrafted features limits the model's adaptability and generalization ability across different spatial regions and time scales. Models that ignore spatial heterogeneity tend to favor popular areas with high traffic volume while neglecting less popular areas with low traffic volume, leading to an incomplete understanding of urban traffic flow and consequently, inaccurate predictions of urban traffic conditions. Summary of the Invention

[0005] This invention provides a traffic prediction method and system based on self-supervised learning to solve the problem of inaccurate prediction of urban traffic conditions.

[0006] In a first aspect, the present invention provides a traffic prediction method based on self-supervised learning, the method comprising the following steps:

[0007] Obtain traffic flow data for the target city;

[0008] The traffic flow data is preprocessed, and the preprocessed traffic flow data is divided into traffic flow training data and traffic flow test data.

[0009] A traffic prediction model is constructed based on a neural network model. The traffic prediction model includes a spatiotemporal encoder, an adaptive data masking module, a multilayer perceptron, a spatial heterogeneity modeling module, and a temporal heterogeneity modeling module.

[0010] The traffic flow training data is input into the traffic prediction model, and the spatiotemporal dependency of the traffic flow data is captured by the spatiotemporal encoder to obtain a similarity matrix and the original spatiotemporal embedding.

[0011] The traffic flow training data is masked using the adaptive data masking module and based on the similarity matrix to obtain enhanced traffic flow data.

[0012] The spatiotemporal dependencies of the enhanced traffic flow data are captured by the spatiotemporal encoder to obtain the enhanced spatiotemporal embedding;

[0013] The original spatiotemporal embedding and the enhanced spatiotemporal embedding are respectively input into the spatial heterogeneity modeling module and the temporal heterogeneity modeling module to obtain the spatial heterogeneity loss function and the temporal heterogeneity loss function;

[0014] Based on the original spatiotemporal embedding, traffic flow is predicted using the multilayer perceptron, and a prediction loss function is obtained.

[0015] The total model loss function is calculated by combining the prediction loss function, the spatial heterogeneity loss function, and the temporal heterogeneity loss function.

[0016] Based on the total loss function of the model, the model parameters of the traffic prediction model are optimized using the backpropagation algorithm to minimize the total loss function of the model.

[0017] The traffic flow test data is input into the traffic prediction model to obtain the traffic prediction results for the target city.

[0018] Optionally, the step of inputting the traffic flow training data into the traffic prediction model and capturing the spatiotemporal dependencies of the traffic flow data through the spatiotemporal encoder to obtain the similarity matrix and the original spatiotemporal embedding includes the following steps:

[0019] The traffic flow training data is input into the traffic prediction model, and the traffic flow training data is processed by the 3D convolutional layer in the spatiotemporal encoder to obtain the training data embedding.

[0020] The training data embedding is processed by the average pooling layer in the spatiotemporal encoder to obtain a similarity matrix;

[0021] The training data embedding is processed using the convolutional long short-term memory network in the spatiotemporal encoder to obtain the original spatiotemporal embedding.

[0022] Optionally, the calculation formula for the training data embedding is as follows:

[0023] X′=ReLU(Conv3D2(ReLU(Conv3D1(X))))

[0024] In the formula: X′ represents the training data embedding, X represents the traffic flow training data, ReLU(·) represents the activation function, Conv3D1(·) represents the first 3D convolutional layer in the spatiotemporal encoder, and Conv3D2(·) represents the second 3D convolutional layer in the spatiotemporal encoder;

[0025] The formula for calculating the similarity matrix is ​​as follows:

[0026] A = Softmax(AvgPool 3D(X′))

[0027] In the formula: A represents the similarity matrix, Softmax(·) represents the normalized exponential function, and AvgPool3D(·) represents the average pooling layer.

[0028] Optionally, the traffic flow data is represented as:

[0029] X = [x t-T+1 x t-T+2 , ..., x t ]

[0030] In the formula: X represents the traffic flow data, Let T represent the traffic flow matrix at the t-th time step in the traffic flow data, where T represents the time range, M represents the number of rows in the grid divided by the target city, and N represents the number of columns in the grid divided by the target city.

[0031] Optionally, the step of using the adaptive data masking module and performing data masking processing on the traffic flow training data based on the similarity matrix to obtain enhanced traffic flow data includes the following steps:

[0032] The adaptive data masking module iterates through all matrix elements in the similarity matrix and calculates the masking probability of each matrix element.

[0033] If the masking probability exceeds a preset probability threshold, then the corresponding matrix element is regarded as the masking target element;

[0034] The traffic flow information of the masked target element within the grid area corresponding to the traffic flow training data is deleted.

[0035] Optionally, the formula for calculating the masking probability is as follows:

[0036]

[0037] In the formula: A t，i，j P represents the matrix element in the i-th row and j-th column at the t-th time step of the similarity matrix. t，i，j This represents the masking probability of the matrix element at time step t.

[0038] Optionally, the step of inputting the original spatiotemporal embedding and the enhanced spatiotemporal embedding into the spatial heterogeneity modeling module and the temporal heterogeneity modeling module, respectively, to obtain the spatial heterogeneity loss function and the temporal heterogeneity loss function includes the following steps:

[0039] The original spatiotemporal embedding and the enhanced spatiotemporal embedding are input into the spatial heterogeneity modeling module;

[0040] The clustering process of the original spatiotemporal embedding and the enhanced spatiotemporal embedding is completed based on the preset clustering prototype, and the original clustering assignment matrix and the enhanced clustering assignment matrix are obtained.

[0041] Combining the original clustering assignment matrix and the enhanced clustering assignment matrix, an optimization task for the spatial heterogeneity modeling module is constructed with the goal of maximizing the similarity between the original spatiotemporal embedding and the enhanced spatiotemporal embedding in the clustering space, and a spatial heterogeneity loss function is determined based on the optimization task of the spatial heterogeneity modeling module.

[0042] The original spatiotemporal embedding and the enhanced spatiotemporal embedding are input into the temporal heterogeneity modeling module, and the temporal heterogeneity loss function is determined based on the contrastive learning method and the bilinear discriminator in the temporal heterogeneity modeling module.

[0043] Optionally, the step of inputting the original spatiotemporal embedding and the enhanced spatiotemporal embedding into the temporal heterogeneity modeling module, and determining the temporal heterogeneity loss function based on the contrastive learning method and according to the bilinear discriminator in the temporal heterogeneity modeling module includes the following steps:

[0044] The original spatiotemporal embedding and the enhanced spatiotemporal embedding are fused into a region-level embedding;

[0045] The city-level embedding is obtained by averaging the regional-level embedding over the spatial dimension of the regional-level embedding.

[0046] The city-level embedding is used as summary information, the region-level embedding at the current time step is used as positive sample embedding, and the region-level embeddings at non-current time steps are all used as negative sample embedding.

[0047] The consistency score between the summary information and the positive sample embedding is obtained using the bilinear discriminant in the temporal heterogeneity modeling module.

[0048] An optimization task for the temporal heterogeneity modeling module is constructed based on the consistency score, and a temporal heterogeneity loss function is determined according to the optimization task of the temporal heterogeneity modeling module.

[0049] Optionally, the step of calculating the total model loss function by combining the prediction loss function, the spatial heterogeneity loss function, and the temporal heterogeneity loss function includes the following steps:

[0050] The total model loss function is obtained by calculating the weighted sum of the prediction loss function, the spatial heterogeneity loss function, and the temporal heterogeneity loss function using the dynamic weighted averaging technique.

[0051] In a second aspect, the present invention also provides a traffic prediction system based on self-supervised learning, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the traffic prediction method based on self-supervised learning as described in the first aspect.

[0052] The beneficial effects of this invention are:

[0053] The traffic prediction model constructed in this invention includes a spatiotemporal encoder, adaptive data masking, spatial heterogeneity modeling, and temporal heterogeneity modeling. The spatiotemporal encoder captures the spatiotemporal dependencies in traffic flow data, while generating a similarity matrix and embedding the prediction results. Adaptive data masking dynamically selects the spatial regions to be masked, enhancing the model's robustness. Spatial and temporal heterogeneity modeling capture the spatial and temporal heterogeneity in traffic flow data, respectively, exploring the complexity of traffic data more deeply and enriching the model's feature space. Therefore, the traffic prediction model constructed in this invention will yield more accurate prediction results. Attached Figure Description

[0054] Figure 1 This is a flowchart illustrating a traffic prediction method based on self-supervised learning in one embodiment of this application.

[0055] Figure 2 This is a flowchart illustrating the framework of a traffic prediction method based on self-supervised learning in one embodiment of this application.

[0056] Figure 3 This is a schematic diagram of the spatial heterogeneity of traffic flow data in a target city, as shown in one embodiment of this application.

[0057] Figure 4 This is a visualization diagram of the temporal heterogeneity of traffic flow data in a target city in one embodiment of this application.

[0058] Figure 5 This is a schematic diagram of the framework flow of the spatial heterogeneity modeling module in one embodiment of this application.

[0059] Figure 6 This is a schematic diagram of the framework flow of the time heterogeneity modeling module in one embodiment of this application. Detailed Implementation

[0060] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0061] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0062] Figure 1 This is a flowchart illustrating a traffic prediction method based on self-supervised learning in one embodiment. It should be understood that, although... Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps. For example Figure 1 As shown, the traffic prediction method based on self-supervised learning disclosed in this invention specifically includes the following steps:

[0063] S101. Obtain traffic flow data for the target city.

[0064] In this context, the target city refers to the city for which traffic forecasting is required. Traffic flow data for the target city can be obtained through its traffic management system. In grid-based traffic forecasting, the target city is divided into many equally sized grids, each representing a spatial region. These regions are the basic units for collecting and analyzing traffic flow data. Each spatial region exhibits inflow and outflow volumes. Inflow represents the amount of traffic entering the spatial region within a specified time interval, encompassing all forms of traffic flow, including cars, bicycles, and pedestrians. Conversely, outflow represents the amount of traffic leaving the spatial region within the same time interval.

[0065] In summary, the traffic flow data is represented as follows:

[0066] X = [x t-T+1 x t-T+2 , ..., x t ]

[0067] In the formula: X represents the traffic flow data, Let T represent the traffic flow matrix at the t-th time step in the traffic flow data, where T represents the time range, M represents the number of rows in the grid divided by the target city, and N represents the number of columns in the grid divided by the target city.

[0068] S102. Preprocess the traffic flow data and divide the preprocessed traffic flow data into traffic flow training data and traffic flow test data.

[0069] The preprocessing steps involve removing outliers and missing values, which may include filling in missing values ​​or using data smoothing techniques to reduce noise. The cleaned data is then divided into two parts: traffic flow training data and traffic flow test data. One part is used to train the model, and the other part is used to evaluate the model's performance.

[0070] S103. Construct a traffic prediction model based on a neural network model.

[0071] Among them, reference Figure 2 The traffic prediction model comprises a spatiotemporal encoder (STEncoder), an adaptive data masking module (ADM), a multilayer perceptron (MLP), a spatial heterogeneity modeling module (SHM), and a temporal heterogeneity modeling module (THM). The spatiotemporal encoder captures the spatiotemporal dependencies in traffic flow data, generating a similarity matrix and embedding the prediction results. The adaptive data masking module dynamically selects the spatial regions to be masked, enhancing the model's robustness. The spatial and temporal heterogeneity modeling modules capture spatial and temporal heterogeneity in traffic flow data, respectively, exploring the complexity of traffic data more deeply and enriching the model's feature space.

[0072] Traffic data contains spatial and temporal heterogeneity, reflecting the variability of traffic flow in different regions and at different time steps. For example, refer to... Figure 3 and Figure 4 , Figure 3 This is a data visualization diagram of traffic flow data for a certain city, specifically showing the inflow heat map of a certain area of ​​the city at 9:00 AM on Sunday, March 1, 2015. Figure 3 This demonstrates the differences in traffic volume between different areas, highlighting the concept of spatial heterogeneity. Spatial heterogeneity can be caused by a variety of factors, including road layout, the location of transportation hubs (such as subway and train stations), the distribution of commercial and residential areas, and events specific to a particular area (such as sporting events and concerts).

[0073] Figure 4 This shows Figure 3Changes in inflows to regions A and B on Sunday, March 1, 2015 and Monday, March 2, 2015. Figure 4 This study demonstrates how traffic patterns in different areas change over time, such as from morning to midnight or from weekends to weekdays, reflecting temporal heterogeneity. These changes are crucial for understanding the dynamics of urban traffic and require sophisticated predictive models to capture this dynamic. The spatial and temporal heterogeneity in traffic data poses challenges to traditional predictive methods, but also provides opportunities to incorporate heterogeneity into the modeling process to improve predictive accuracy.

[0074] S104. Input the traffic flow training data into the traffic prediction model, capture the spatiotemporal dependence of the traffic flow data through the spatiotemporal encoder, and obtain the similarity matrix and the original spatiotemporal embedding.

[0075] Among them, such as Figure 2 As shown, Figure 2 In this context, A represents the similarity matrix, and H represents the original spatiotemporal embedding.

[0076] S105. Using an adaptive data masking module and based on a similarity matrix, the traffic flow training data is masked to obtain enhanced traffic flow data.

[0077] S106. Enhanced spatiotemporal embedding is obtained by capturing the spatiotemporal dependence of traffic flow data through a spatiotemporal encoder.

[0078] Among them, such as Figure 2 As shown, Figure 2 middle This is known as enhanced spatiotemporal embedding.

[0079] S107. Input the original spatiotemporal embedding and the enhanced spatiotemporal embedding into the spatial heterogeneity modeling module and the temporal heterogeneity modeling module, respectively, to obtain the spatial heterogeneity loss function and the temporal heterogeneity loss function.

[0080] S108. Based on the original spatiotemporal embedding and through a multilayer perceptron, traffic flow is predicted, and the prediction loss function is obtained.

[0081] S109. The total loss function of the model is calculated by combining the prediction loss function, the spatial heterogeneity loss function, and the temporal heterogeneity loss function.

[0082] S110. Optimize the model parameters of the traffic prediction model based on the total loss function of the model and through the backpropagation algorithm to minimize the total loss function of the model.

[0083] S111. Input the traffic flow test data into the traffic prediction model to obtain the traffic prediction results for the target city.

[0084] In one embodiment, step S104 specifically includes the following steps:

[0085] Traffic flow training data is input into the traffic prediction model, and the traffic flow training data is processed by the 3D convolutional layer in the spatiotemporal encoder to obtain the training data embedding.

[0086] The similarity matrix is ​​obtained by processing the training data embedding through the average pooling layer in the spatiotemporal encoder.

[0087] The training data embedding is processed using a convolutional long short-term memory network in the spatiotemporal encoder to obtain the original spatiotemporal embedding.

[0088] In this embodiment, the spatiotemporal encoder aims to effectively capture the spatiotemporal dependencies of traffic flow data, providing rich spatiotemporal embeddings for subsequent modules. The spatiotemporal encoder consists of several key layers, each uniquely contributing to the model's overall ability to process and interpret traffic flow data. First, the traffic data is processed through two 3D convolutional layers. These 3D convolutional layers can simultaneously handle spatial and temporal dimensions, allowing interaction between adjacent regions and adjacent time steps, extracting features reflecting the dynamics of traffic flow. The formula for embedding the training data after processing the continuous traffic flow data is as follows:

[0089] X′=ReLU(Conv3D2(ReLU(Conv3D1(X))))

[0090] In the formula: X′ represents the training data embedding, X represents the traffic flow training data, ReLU(·) represents the activation function, Conv3D1(·) represents the first 3D convolutional layer in the spatiotemporal encoder, and Conv3D2(·) represents the second 3D convolutional layer in the spatiotemporal encoder.

[0091] Next, a crucial function of the spatiotemporal encoder is the calculation of the similarity matrix, derived from the training data embeddings. This matrix is ​​used for adaptive data masking, enhancing the model's training data by emphasizing the similarities between different traffic patterns. The formula for calculating the similarity matrix is ​​as follows:

[0092] A = Softmax(AvgPool3D(X′))

[0093] In the formula: A represents the similarity matrix, Softmax(·) represents the normalized exponential function, which is used to emphasize the relative importance of different time steps in traffic data, and AvgPool3D(·) represents the average pooling layer.

[0094] The core of the spatiotemporal encoder is the ConvLSTM layer, chosen for its adeptness at capturing spatiotemporal dependencies. LSTM, short for Long Short-Term Memory networks, is a variant of ordinary RNNs and effectively addresses the problem of long-term dependencies. The core of LSTM is the cell state. Input information passes through the cell, interacting only minimally with other solid lines, representing some linear transformations. This allows the input information to be preserved relatively completely, meaning it retains long-term memory. LSTM selectively retains information through a gating mechanism. The gating structure controls the amount of information passing through the cell. It essentially performs a linear transformation on the input information and then passes it through a sigmoid layer, ultimately converting the input into a coefficient vector with values ​​ranging from 0 to 1, which can be understood as the proportion of information retained. A value of 0 indicates that all corresponding information is discarded, while a value of 1 indicates that all corresponding information is retained. Unlike standard LSTMs that only process time-series data, the Convolutional Long Short-Term Memory network (ConvLSTM) extends its capabilities to the spatial dimension, making it particularly suitable for traffic prediction tasks where spatial relationships are crucial. The detailed formula for the convolutional long short-term memory network layer is as follows:

[0095]

[0096]

[0097]

[0098]

[0099]

[0100] In one embodiment, step S105 specifically includes the following steps:

[0101] The adaptive data masking module iterates through all matrix elements in the similarity matrix and calculates the masking probability of each matrix element.

[0102] If the masking probability exceeds the preset probability threshold, the corresponding matrix element will be regarded as the masking target element.

[0103] Remove the traffic flow information from the grid area corresponding to the target element in the traffic flow training data.

[0104] In this embodiment, the adaptive data masking module is a key component in enhancing the robustness and generalization ability of the prediction model. Unlike traditional random masking techniques, the adaptive data masking module in this invention utilizes the similarity matrix A obtained from the spatiotemporal encoder to design a targeted data masking strategy. This strategy ensures that the enhancement focuses on the most informative parts of the traffic flow data, thereby enabling the model to simulate realistic and challenging traffic scenarios.

[0105] Given a similarity matrix A, representing the normalized importance of each spatial region at each time step. The ultimate goal is to mask regions whose importance is inversely proportional to their similarity scores, meaning regions with lower similarity scores are more likely to be masked. This is achieved by calculating a masking probability distribution from A, where a given spatial region has a higher probability of being masked if its corresponding similarity score is low.

[0106] In summary, we can first use the adaptive data masking module to traverse all matrix elements in the similarity matrix and calculate the masking probability of each matrix element. The specific calculation formula is as follows:

[0107]

[0108] In the formula: A t，i，j P represents the matrix element in the i-th row and j-th column at the t-th time step in the similarity matrix. t，i，j This represents the masking probability of a matrix element at time step t.

[0109] The actual masking operation is based on P t，i，j The selection of masked regions can be achieved by setting a probability threshold. Specifically, if the masking probability exceeds the preset threshold, the corresponding matrix element is considered the masking target element, and the traffic flow information within the corresponding grid area of ​​the masking target element in the traffic flow training data is deleted. The total number of masked regions is determined by a predefined masking ratio. Deleting traffic flow information specifically means setting the inflow and outflow volumes of the selected spatial region to zero, simulating the absence of traffic flow information in these regions. This method not only challenges the model to make predictions in the absence of certain data but also encourages the model to use its understanding of spatiotemporal dependencies to fill in the missing information.

[0110] In one embodiment, step S107 specifically includes the following steps:

[0111] The original spatiotemporal embedding and the enhanced spatiotemporal embedding are input into the spatial heterogeneity modeling module;

[0112] Based on the preset clustering prototype, the clustering process of original spatiotemporal embedding and enhanced spatiotemporal embedding is completed to obtain the original clustering assignment matrix and the enhanced clustering assignment matrix;

[0113] By combining the original clustering assignment matrix and the enhanced clustering assignment matrix, an optimization task for the spatial heterogeneity modeling module is constructed with the goal of maximizing the similarity between the original spatiotemporal embedding and the enhanced spatiotemporal embedding in the clustering space. The spatial heterogeneity loss function is then determined based on the optimization task of the spatial heterogeneity modeling module.

[0114] The original spatiotemporal embedding and the enhanced spatiotemporal embedding are input into the temporal heterogeneity modeling module. The temporal heterogeneity loss function is determined based on the contrastive learning method and the bilinear discriminator in the temporal heterogeneity modeling module.

[0115] In this embodiment, refer to Figure 5 The spatial heterogeneity modeling module is a crucial component of the model, aiming to capture the potential spatial heterogeneity in traffic data through self-supervised signals. To achieve this goal, a self-supervised learning task based on soft clustering can be designed, such as... Figure 5 As shown. Specifically, the embeddings of different spatial regions are mapped onto prototypes corresponding to different urban functions (such as residential areas, office areas, and transportation hubs) to form cluster prototypes. Previously, the embeddings of the original data and the embeddings of the augmented data, namely H and [other data], were obtained through a spatiotemporal encoder. spatial region r i，j The original spatiotemporal embedding and the enhanced spatiotemporal embedding are respectively represented as h i，j and Suppose the clustering prototype contains K clusters, specifically represented as {c1, c2, ..., c3}. K The clustering process for enhanced spatiotemporal embedding is implemented through the following formula:

[0116]

[0117] In the formula: Represents spatial region r i，j Enhanced spatiotemporal embedding With prototype c k The similarity score between them. Therefore, the spatial region r i，j The clustering assignment can be represented as: The enhanced clustering assignment matrix can be obtained by combining the clustering assignments of all spatial regions. Similarly, the clustering process of the original spatiotemporal embedding is implemented through the following formula:

[0118]

[0119] In the formula: It is the original spatiotemporal embedding h i，j With prototype c k The similarity score between them. Therefore, the spatial region r i，j The clustering assignment can be represented as: The original clustering assignment matrix can be obtained by combining the clustering assignments of all spatial regions.

[0120] The optimization task of constructing a spatial heterogeneity modeling module with the goal of maximizing the similarity between the original spatiotemporal embedding and the enhanced spatiotemporal embedding in the clustering space can be expressed by the following formula:

[0121]

[0122] In the formula: l∈[1,2,...,K], τ represents the temperature parameter, which is used to control the sharpness of the distribution of the Softmax function output.

[0123] Finally, the sum of the loss functions for all spatial regions is used as the final loss for spatial heterogeneity, as follows:

[0124]

[0125] By minimizing the cross-entropy of the original spatiotemporal embedding and the enhanced spatiotemporal embedding in the cluster space, the two embeddings are made as close as possible in cluster assignment.

[0126] In the above process, the enhanced clustering assignment matrix and the original clustering assignment matrix are used as self-supervised signals for spatial heterogeneity modeling. Then, two problems need to be addressed to ensure that the regional features conform to the true distribution of urban space: (1) ensuring that the sum of the clustering assignment matrices for each region is 1; (2) avoiding the situation where all regions receive the same assignment. To solve these two problems, in one implementation, the Sinkhorn algorithm, an entropy-based regularization optimization method, can be introduced. It is used to adjust the clustering assignment matrix to satisfy certain normalization conditions, namely, the sum of the assignments of each spatial region across all clusters is 1, and the sum of the assignments of each cluster across all spatial regions is also 1. Through alternating normalization of the two dimensions of spatial regions and clusters, the Sinkhorn algorithm can achieve a balanced clustering assignment strategy. Specifically, the Sinkhorn algorithm can be applied to... and The original allocation matrix is ​​then replaced with the result obtained from the algorithm.

[0127] In one implementation, the step of inputting the original spatiotemporal embedding and the enhanced spatiotemporal embedding into the temporal heterogeneity modeling module, and determining the temporal heterogeneity loss function based on the contrastive learning method and the bilinear discriminator in the temporal heterogeneity modeling module, specifically includes the following steps:

[0128] The original spatiotemporal embedding and the enhanced spatiotemporal embedding are fused into a region-level embedding;

[0129] The city-level embedding is obtained by averaging the regional-level embeddings along the spatial dimension of the regional-level embeddings.

[0130] City-level embeddings are used as summary information, regional-level embeddings at the current time step are used as positive sample embeddings, and regional-level embeddings at non-current time steps are all used as negative sample embeddings.

[0131] The consistency score between the summary information and the positive sample embedding is obtained by using the bilinear discriminator in the temporal heterogeneity modeling module.

[0132] An optimization task for the temporal heterogeneity modeling module is constructed based on the consistency score, and the temporal heterogeneity loss function is determined according to the optimization task of the temporal heterogeneity modeling module.

[0133] In this embodiment, refer to Figure 6 The temporal heterogeneity modeling module is based on contrastive learning, capturing changes in traffic patterns at different time steps by comparing the consistency between the summary information of spatial regions and positive and negative samples. To incorporate temporal heterogeneity into the prediction model, a self-supervised learning task based on contrastive learning can be designed, such as... Figure 6 As shown. This task aims to enhance the model's understanding of temporal dynamics by identifying and capturing changes in traffic patterns at different time steps through contrastive learning. First, the spatial region r... i，j The original spacetime embedding h at time step t i，j and enhanced spatiotemporal embedding The fusion process yields a region-level embedding. t，i，j The specific formula is as follows:

[0134]

[0135] In the formula: w1 and w2 represent learnable weights, and ⊙ represents element-level multiplication.

[0136] Then, based on region-level embedding u t，i，j Generate city-level embeddings t Specifically, it can be used for u t，i，j The average is taken over its spatial dimension, and the Sigmoid activation function is applied to obtain s. t The specific formula is as follows:

[0137]

[0138] Subsequently, the city level is embedded in s t As summary information, region-level embedding u t，i，j As positive sample embeddings, and region-level embeddings at other time steps, u t′，i，j As negative sample embeddings. To evaluate summary information s tTo maintain consistency with the positive and negative sample embeddings, a bilinear discriminant is introduced. (Summary information s) t With positive sample embedding h t，i，j The consistency score obtained by the bilinear discriminant can be calculated using the following formula:

[0139]

[0140] In the formula: It is a learnable weight matrix, and b represents the bias term.

[0141] The temporal heterogeneity modeling module can compare summary information s t With positive sample embedding h t，i，j and negative sample embedding h t′，i，j The consistency score is used for optimization, and the specific formula is as follows:

[0142]

[0143] Based on the above formula, the sum of the loss functions for all spatial regions is taken as the final loss, and the specific formula is as follows:

[0144]

[0145] This positive and negative sample contrast learning mechanism ensures that the prediction results are consistent with the traffic patterns at specific time steps in the city, while distinguishing traffic patterns at other time steps and learning temporal heterogeneity.

[0146] In one embodiment, step S108 specifically includes the following steps:

[0147] Traffic flow can be predicted using a multilayer perceptron (MLP), as expressed by the following formula:

[0148]

[0149] In the formula: Represents spatial region r i，j Traffic flow forecast at time step t+1.

[0150] Then, the prediction loss function is calculated using the mean absolute error, as shown in the following formula:

[0151]

[0152] In the formula: Let λ represent the prediction loss function, and let y represent the hyperparameters. t+1，i，j Represents spatial region r i，j The actual traffic flow value at time step t+1.

[0153] In one embodiment, step S109 specifically includes the following steps:

[0154] The total loss function of the model is obtained by calculating the weighted sum of the prediction loss function, spatial heterogeneity loss function, and temporal heterogeneity loss function using the dynamic weighted averaging technique.

[0155] In this embodiment, the total loss function It is a weighted sum of three loss functions, and the specific formula is as follows:

[0156]

[0157] In the formula, α, β, and γ represent weights. To adapt to the scale and complexity of different tasks, a dynamic weight adjustment mechanism, namely Dynamic Weight Averaging (DWA), is adopted. Initially, the weights α, β, and γ are set to [1, 1, 1], giving each loss equal importance. DWA recalibrates the weights according to the relative learning progress of each task, ensuring balanced optimization among different modules.

[0158] The present invention also discloses a traffic prediction system based on self-supervised learning, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the traffic prediction method based on self-supervised learning as described in any of the above embodiments.

[0159] The processor can be a central processing unit (CPU). Of course, depending on the actual use, it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc., and this application does not limit it in this regard.

[0160] The memory can be an internal storage unit of a computer device, such as a hard disk or RAM, or an external storage device, such as a plug-in hard disk, smart memory card (SMC), secure digital card (SD), or flash memory card (FC) provided on the computer device. Furthermore, the memory can be a combination of internal storage units and external storage devices of a computer device. The memory is used to store computer programs and other programs and data required by the computer device. The memory can also be used to temporarily store data that has been output or will be output. This application does not limit this.

[0161] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of protection of this application is limited to these examples; under the concept of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of one or more embodiments of this application as described above, which are not provided in detail for the sake of brevity.

[0162] One or more embodiments in this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of this application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments in this application should be included within the protection scope of this application.

Claims

1. A traffic prediction method based on self-supervised learning, characterized in that, Includes the following steps: Obtain traffic flow data for the target city; The traffic flow data is preprocessed, and the preprocessed traffic flow data is divided into traffic flow training data and traffic flow test data. A traffic prediction model is constructed based on a neural network model. The traffic prediction model includes a spatiotemporal encoder, an adaptive data masking module, a multilayer perceptron, a spatial heterogeneity modeling module, and a temporal heterogeneity modeling module. The traffic flow training data is input into the traffic prediction model, and the spatiotemporal dependency of the traffic flow data is captured by the spatiotemporal encoder to obtain a similarity matrix and the original spatiotemporal embedding. The traffic flow training data is masked using the adaptive data masking module and based on the similarity matrix to obtain enhanced traffic flow data. The spatiotemporal dependencies of the enhanced traffic flow data are captured by the spatiotemporal encoder to obtain the enhanced spatiotemporal embedding; The original spatiotemporal embedding and the enhanced spatiotemporal embedding are respectively input into the spatial heterogeneity modeling module and the temporal heterogeneity modeling module to obtain the spatial heterogeneity loss function and the temporal heterogeneity loss function; Based on the original spatiotemporal embedding, traffic flow is predicted using the multilayer perceptron, and a prediction loss function is obtained. The total model loss function is calculated by combining the prediction loss function, the spatial heterogeneity loss function, and the temporal heterogeneity loss function, specifically including the following steps: The total model loss function is obtained by calculating the weighted sum of the prediction loss function, the spatial heterogeneity loss function, and the temporal heterogeneity loss function using the dynamic weight averaging technique. Based on the total loss function of the model, the model parameters of the traffic prediction model are optimized using the backpropagation algorithm to minimize the total loss function of the model. The traffic flow test data is input into the traffic prediction model to obtain the traffic prediction results for the target city; The step of inputting the traffic flow training data into the traffic prediction model and capturing the spatiotemporal dependencies of the traffic flow data through the spatiotemporal encoder to obtain the similarity matrix and the original spatiotemporal embedding includes the following steps: The traffic flow training data is input into the traffic prediction model, and the traffic flow training data is processed by the 3D convolutional layer in the spatiotemporal encoder to obtain the training data embedding. The training data embedding is processed by the average pooling layer in the spatiotemporal encoder to obtain a similarity matrix; The training data embedding is processed using the convolutional long short-term memory network in the spatiotemporal encoder to obtain the original spatiotemporal embedding. The calculation formula for the training data embedding is as follows: wherein: denotes the training data embedding, denotes the traffic flow training data, denotes an activation function, denotes a first 3D convolutional layer in the spatio-temporal encoder, denotes a second 3D convolutional layer in the spatio-temporal encoder; The formula for calculating the similarity matrix is ​​as follows: In the formula: Represents the similarity matrix, Represents the normalized exponential function, This refers to the average pooling layer.

2. The traffic prediction method based on self-supervised learning according to claim 1, characterized in that, The traffic flow data is represented as follows: In the formula: This refers to the traffic flow data. Indicates the first of the traffic flow data Traffic flow matrix at each time step Indicates a time range. This indicates the number of rows in the grid that divides the target city. This indicates the number of columns in the grid that the target city is divided into.

3. The traffic prediction method based on self-supervised learning according to claim 2, characterized in that, The step of using the adaptive data masking module and performing data masking processing on the traffic flow training data based on the similarity matrix to obtain enhanced traffic flow data includes the following steps: The adaptive data masking module iterates through all matrix elements in the similarity matrix and calculates the masking probability of each matrix element. If the masking probability exceeds a preset probability threshold, then the corresponding matrix element is regarded as the masking target element; The traffic flow information of the masked target element within the grid area corresponding to the traffic flow training data is deleted.

4. The traffic prediction method based on self-supervised learning according to claim 3, characterized in that, The formula for calculating the masking probability is as follows: In the formula: In the similarity matrix, the first... The time step Line 1 Column matrix elements, Indicates that the matrix element is in the first position. The masking probability at each time step.

5. The traffic prediction method based on self-supervised learning according to claim 2, characterized in that, The step of inputting the original spatiotemporal embedding and the enhanced spatiotemporal embedding into the spatial heterogeneity modeling module and the temporal heterogeneity modeling module, respectively, to obtain the spatial heterogeneity loss function and the temporal heterogeneity loss function includes the following steps: The original spatiotemporal embedding and the enhanced spatiotemporal embedding are input into the spatial heterogeneity modeling module; The clustering process of the original spatiotemporal embedding and the enhanced spatiotemporal embedding is completed based on the preset clustering prototype, and the original clustering assignment matrix and the enhanced clustering assignment matrix are obtained. Combining the original clustering assignment matrix and the enhanced clustering assignment matrix, an optimization task for the spatial heterogeneity modeling module is constructed with the goal of maximizing the similarity between the original spatiotemporal embedding and the enhanced spatiotemporal embedding in the clustering space, and a spatial heterogeneity loss function is determined based on the optimization task of the spatial heterogeneity modeling module. The original spatiotemporal embedding and the enhanced spatiotemporal embedding are input into the temporal heterogeneity modeling module, and the temporal heterogeneity loss function is determined based on the contrastive learning method and the bilinear discriminator in the temporal heterogeneity modeling module.

6. The traffic prediction method based on self-supervised learning according to claim 5, characterized in that, The step of inputting the original spatiotemporal embedding and the enhanced spatiotemporal embedding into the temporal heterogeneity modeling module, and determining the temporal heterogeneity loss function based on the contrastive learning method and the bilinear discriminator in the temporal heterogeneity modeling module includes the following steps: The original spatiotemporal embedding and the enhanced spatiotemporal embedding are fused into a region-level embedding; The city-level embedding is obtained by averaging the regional-level embedding over the spatial dimension of the regional-level embedding. The city-level embedding is used as summary information, the region-level embedding at the current time step is used as positive sample embedding, and the region-level embeddings at non-current time steps are all used as negative sample embedding. The consistency score between the summary information and the positive sample embedding is obtained using the bilinear discriminant in the temporal heterogeneity modeling module. An optimization task for the temporal heterogeneity modeling module is constructed based on the consistency score, and a temporal heterogeneity loss function is determined according to the optimization task of the temporal heterogeneity modeling module.

7. A traffic prediction system based on self-supervised learning, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the traffic prediction method based on self-supervised learning as described in any one of claims 1 to 6.

Images