Traffic Flow Prediction Method, Device, Computer Equipment and Storage Medium

Abstract

The present invention discloses a traffic flow prediction method, device, computer device and storage medium, including: constructing a real urban traffic road scene by using a digital twin simulation platform and collecting twin traffic data; forming a hybrid dataset containing twins and reality with a mixing ratio α for the twin traffic data and real traffic data, and then performing multi-time dimension fusion to obtain a first fusion feature; respectively extracting features based on weather data and traffic light information to obtain weather features and traffic information features; performing feature fusion on the first fusion feature, weather features and traffic information features to obtain a second fusion feature; performing traffic flow prediction according to the second fusion feature to obtain a traffic flow prediction result. Using the present invention improves the accuracy of traffic flow prediction in complex environments.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation, and in particular, to a traffic flow prediction method, device, computer device, and storage medium. Background Art

[0002] With the acceleration of the urbanization process, urban traffic flow prediction has become a key part of the intelligent transportation system. As one of the most important functions of today's intelligent transportation system, traffic prediction uses historical traffic data on roads to predict the future situation of the traffic network, which helps to improve the efficiency of the traffic system, improve traffic management, and optimize the traffic travel experience. Traditional traffic flow prediction methods mainly rely on historical traffic data and use techniques such as statistical analysis and machine learning to analyze and predict traffic flow trends.

[0003] In the process of implementing the present invention, the inventors realized that the existing technology has at least the following technical problems: The existing methods highly rely on the quality and quantity of historical data. In new cities or regions, due to the lack of sufficient historical data, the prediction accuracy will drop significantly, making it difficult to meet the needs of practical applications. At the same time, urban traffic flow is affected by various non-linear factors, such as seasonal changes, special events, weather conditions, etc. It is difficult for the existing methods to comprehensively consider the complexity and variability of these factors, thus affecting the prediction accuracy. Summary of the Invention

[0004] Embodiments of the present invention provide a traffic flow prediction method, device, computer device, and storage medium to improve the accuracy of traffic flow prediction.

[0005] To solve the above technical problems, an embodiment of the present application provides a traffic flow prediction method, including:

[0006] Using a digital twin simulation platform to construct a real urban traffic road scene and collect twin traffic data;

[0007] Forming a mixed dataset containing twins and reality with a mixing ratio α for the twin traffic data and the real traffic data, and then performing multi-time dimension fusion to obtain a first fusion feature;

[0008] Respectively extracting features based on weather data and traffic light information to obtain weather features and traffic information features;

[0009] Performing feature fusion on the first fusion feature, the weather feature, and the traffic information feature to obtain a second fusion feature;

[0010] Performing traffic flow prediction based on the second fusion feature to obtain a traffic flow prediction result.

[0011] Optionally, the collection of twin traffic data includes:

[0012] Through the API interface of Carla, the status and duration of traffic lights and the weather data of the current simulation scenario are collected in real time;

[0013] Using the scenario generation ability provided by Carla, by directionally adjusting parameters such as traffic flow, weather conditions, and signal timing, ordinary scenarios and target-specific scenarios based on different urban traffic environments are generated. The target-specific scenarios include simulated harsh environment scenarios, urban morning rush hour scenarios, and urban road accident scenarios;

[0014] Based on various scenarios, traffic data is collected using the built-in sensors of the Carla simulation platform. By changing the number and type of vehicles, weather, and traffic light configuration schemes in the scenario, a realistic traffic scenario is simulated. The point cloud data obtained from the RGB camera and lidar sensors is used to record the trajectory data of vehicles in the scenario, including the position coordinates, speed, and deflection angle of vehicles within each timestamp, to obtain the twin traffic data.

[0015] Optionally, the twin traffic data and the real traffic data are formed into a mixed dataset containing twins and reality in a mixing ratio of α, and then fused in multiple time dimensions to obtain the first fusion features, including:

[0016] The following formula is used to represent the traffic data features in different time dimensions:

[0017] ;

[0018] ;

[0019] ;

[0020] where respectively represent the number of timestamps selected from hours, days, and weeks, and predictions are made in three dimensions at intervals of h timestamps, d timestamps, and w timestamps;

[0021] The traffic data features in the three dimensions are normalized to obtain the normalized feature matrix ;

[0022] Using a feature pyramid network, the normalized feature matrix is fused to obtain the first fusion feature.

[0023] Optionally, the use of a feature pyramid network to fuse the normalized feature matrix to obtain the first fusion feature includes:

[0024] By setting different time extraction windows, the normalized feature matrix The moment feature map is obtained through the extraction layer ResNet network of the feature pyramid network respectively , the day feature map , the cycle feature map , and the feature map extraction formula is as follows:

[0025] ;

[0026] where represents the feature map extracted by the ResNet network at the L-th layer, and L is 0, 1, 2, is the ResNet feature map extraction function, is the weight matrix of each layer;

[0027] The moment feature map , the day feature map , and the cycle feature map are respectively used as the layer, layer and layer of the feature pyramid network;

[0028] Design Three layers are used as the feature fusion layer, and its fusion formula is as follows:

[0029] ;

[0030] where represents the fusion result of the previous layer, is , is the feature map of the extraction layer corresponding to the current fusion layer, is the Hadamard product operation;

[0031] The obtained fusion result is converted into a feature vector through the global average pooling layer to retain the existing feature information in the feature map, and the first fusion feature is obtained.

[0032] Optionally, the feature fusion of the first fusion feature, the weather feature and the traffic information feature to obtain the second fusion feature includes:

[0033] The weather feature matrix is passed through the above ResNet network to obtain the feature map .

[0034] For , the ROIAlign method is used for weather feature extraction to obtain ;

[0035] Calculate The attention matrix between the flow feature map in the feature fusion layer of the feature pyramid , the attention feature map based on the flow feature , the calculation formula is as follows:

[0036] ;

[0037] ;

[0038] where represents the feature map input to the i-th fusion layer, i = 0, 1, 2, and C represents the feature dimension of the current self-attention fusion;

[0039] Adding the attention feature map based on the flow feature to the original to obtain the fusion feature of the upper layer ;

[0040] Using the fusion feature F of the upper layer as the input feature of the next layer's feature fusion layer of the feature pyramid for iterative fusion.

[0041] Optionally, the traffic flow prediction based on the second fusion feature to obtain the traffic flow prediction result includes:

[0042] Using the adjacency matrix and the second fusion feature as the input of the generator in the generative adversarial network, and judging the authenticity of the prediction result based on the discriminator in the generative adversarial network;

[0043] When the discriminator determines that the prediction result is true, obtaining the traffic flow prediction result.

[0044] To solve the above technical problems, an embodiment of the present application further provides a traffic flow prediction device, including:

[0045] A data collection module for constructing a real urban traffic road scene using a digital twin simulation platform and collecting twin traffic data;

[0046] A first fusion module for forming a mixed dataset containing twins and reality with a mixing ratio α for the twin traffic data and the real traffic data, and then performing multi-time dimension fusion to obtain a first fusion feature;

[0047] A feature extraction module for respectively extracting features based on weather data and traffic light information to obtain weather features and traffic information features;

[0048] A second fusion module for performing feature fusion on the first fusion feature, the weather feature, and the traffic information feature to obtain a second fusion feature;

[0049] A traffic flow prediction module, configured to perform traffic flow prediction based on the second fused feature to obtain a traffic flow prediction result.

[0050] Optionally, the first fusion module includes:

[0051] A data characterization unit, configured to represent traffic data features in different time dimensions by the following formula:

[0052] ;

[0053] ;

[0054] ;

[0055] where respectively represent the number of timestamps selected from hours, days, and weeks, and predictions are made in three dimensions at intervals of h timestamps, d timestamps, and w timestamps;

[0056] A normalization unit, configured to perform data normalization on the traffic data features in the three dimensions to obtain a normalized feature matrix ;

[0057] A fusion unit, configured to use a feature pyramid network to fuse the normalized feature matrix to obtain a first fused feature.

[0058] To solve the above technical problem, an embodiment of the present application further provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above traffic flow prediction method are implemented.

[0059] To solve the above technical problem, an embodiment of the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above traffic flow prediction method are implemented.

[0060] The traffic flow prediction method, device, computer device, and storage medium provided by the embodiments of the present invention construct a real urban traffic road scene through a digital twin simulation platform and collect twin traffic data; form a mixed dataset containing twins and reality with a mixing ratio α for the twin traffic data and real traffic data, and then perform multi-time dimension fusion to obtain a first fusion feature; respectively perform feature extraction based on weather data and traffic light information to obtain weather features and traffic information features; perform feature fusion on the first fusion feature, weather features, and traffic information features to obtain a second fusion feature; perform traffic flow prediction based on the second fusion feature to obtain a traffic flow prediction result. It realizes multi-level fusion processing of multi-dimensional traffic data, and further fuses with weather data and traffic signal data to obtain multi-dimensional feature data, improving the accuracy of traffic flow prediction in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0062] Figure 1 is an exemplary system architecture diagram to which the present application can be applied;

[0063] Figure 2 is a flowchart of an embodiment of the traffic flow prediction method of the present application;

[0064] Figure 3 is a schematic diagram of feature fusion using a feature pyramid network in an embodiment of the present application;

[0065] Figure 4 is a schematic structural diagram of an embodiment of the traffic flow prediction device according to the present application;

[0066] Figure 5 is a schematic structural diagram of an embodiment of the computer device according to the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0067] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" and any variations thereof in the specification and claims of this application and the above drawings are intended to cover non-exclusive inclusion. The terms "first", "second", etc. in the specification and claims of this application or the above drawings are used to distinguish different objects and not to describe a specific order.

[0068] References to "embodiments" in this document mean that a particular feature, structure, or characteristic described in connection with the embodiments can be included in at least one embodiment of this application. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0069] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0070] Please refer to Figure 1 , as Figure 1 shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0071] Users can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc.

[0072] The terminal devices 101, 102, and 103 can be various electronic devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV) players, laptop computers, desktop computers, and so on.

[0073] The server 105 can be a server that provides various services, such as a background server that supports the pages displayed on the terminal devices 101, 102, and 103.

[0074] It should be noted that the traffic flow prediction method provided in the embodiments of the present application is executed by the server. Correspondingly, the traffic flow prediction device is set in the server.

[0075] It should be understood that Figure 1 the numbers of the terminal devices, the network, and the server in

[0076] are merely illustrative. According to the implementation requirements, there can be any number of terminal devices, networks, and servers. The terminal devices 101, 102, and 103 in the embodiments of the present application can specifically correspond to the application systems in actual production. Figure 2 Figure 2 Please refer to Figure 1 which shows a traffic flow prediction method provided by an embodiment of the present invention. Taking the application of this method in the

[0077] S201: Use a digital twin simulation platform to build a real urban traffic road scene and collect twin traffic data.

[0078] In a specific optional implementation manner, collecting twin traffic data includes:

[0079] Through the API interface of Carla, real-time collect the status and duration of traffic lights and the weather data of the current simulation scene;

[0080] Utilize the scene generation ability provided by Carla, and generate ordinary scenes and target specific scenes based on different urban traffic environments by directionally adjusting parameters such as traffic flow, weather conditions, and signal timing. The target specific scenes include simulated harsh environment scenes, urban morning rush hour scenes, and urban road accident scenes;

[0081] ​Based on various scenarios, use the sensors built into the Carla simulation platform to collect traffic data. Simulate real traffic scenarios by changing the number and type of vehicles, weather, and traffic light configuration in the scenario. Use the point cloud data obtained from RGB cameras and lidar sensors to record the trajectory data of vehicles in the scenario, including the position coordinates, speed, and deflection angle of vehicles at each timestamp, to obtain twin traffic data.

[0082] Specifically, the digital twin module in this embodiment constructs a real urban traffic road scenario by using a digital twin simulation platform based on Carla, sets the sensor objects of the simulation platform, and communicates with the python API interface to collect traffic data for the twin scenario.

[0083] Furthermore, the steps to construct a real urban traffic road scenario based on the digital twin simulation platform of Carla are as follows:

[0084] Step 1: Map data collection. Collect high-precision real-time map data of the current city, including road networks, traffic signs, etc.; obtain regional radar data to get more accurate 3D spatial information.

[0085] Step 2: Scenario generation and optimization. In the Roadrunner simulation tool, construct a basic road network scenario according to the above map data set, including intersections, viaducts, sidewalks, traffic signs, etc.; export the files generated by roadrunner into FBX format or datasmith format scenario files compatible with the Carla simulation platform to generate a twin scenario of the urban road based on Carla. Add fine lighting effects, traffic light signal systems, sensor devices, initial traffic twin behavior models of vehicles and pedestrians, etc. in this scenario to enhance the visual authenticity of the urban traffic road scenario.

[0086] Step 3: Scenario simulation. Based on the Python API provided by Carla, develop a client program to interact with the simulation environment to achieve real-time control and optimization of dynamic factors such as weather, traffic lights, and areas of interest. By comparing the simulation results with actual traffic data, continuously optimize the parameters of the simulation model to improve the accuracy and credibility of the digital twin scenario.

[0087] Furthermore, the collection of traffic data for the traffic twin scenario includes the following steps:

[0088] Step 1: Use the sensors built in the Carla simulation platform to collect traffic data. By changing settings such as the number and type of vehicles in the scenario, weather, and traffic light timing plans, simulate real traffic scenarios. Use the point cloud data obtained from RGB cameras and lidar sensors to record the trajectory data of vehicles in the scenario, including the position coordinates, speed, and deflection angle of vehicles at each timestamp.

[0089] Step 2: Through the API interface of Carla, collect data such as the status and duration of traffic lights in real time, and the weather information of the current simulation scenario.

[0090] Step 3: Utilize the scenario generation ability provided by Carla. By directionally adjusting parameters such as traffic flow, weather conditions, and signal timing, generate target specific scenarios based on different urban traffic environments, including simulating harsh environment scenarios, urban morning rush hour scenarios, urban road accident scenarios, etc. Process and integrate the collected raw data to form a structured traffic dataset.

[0091] S202: Form a hybrid dataset containing digital twins and real data with a mixing ratio of α for the digital twin traffic data and real traffic data, and then perform multi-time dimension fusion to obtain the first fusion feature.

[0092] Specifically, in this embodiment, a multi-source data fusion module is adopted to perform multi-level fusion on the data collected by the digital twin module and real traffic data. According to the simulated vehicle position data of the above traffic digital twin platform, calculate the number of vehicles in each intersection area within each time interval to obtain traffic flow data , obtain the characteristic data of the current duration of the simulated traffic light , the traffic signal status data S of the target intersection, and the scenario weather parameter data , obtain the urban intersection traffic flow data obtained by real urban sensors and use for dataset augmentation, and finally obtain the final fusion feature matrix through a multi-source fusion model .

[0093] Among them, the mixing ratio α is defined according to actual application needs, such as 3:2, and no specific limitation is made here.

[0094] Furthermore, the simulated traffic flow data obtained by the digital twin is designed according to the traffic congestion situation in real cities, by setting different weather rendering options (such as rainy days, snowy days, foggy days, etc.), the regional population flow in different urban areas, and the morning rush hour scenarios that occur in real traffic. This part of the data can fully reflect the changes in urban area traffic flow under today's special traffic scenarios.

[0095] Furthermore, the weather data based on the simulation platform in this embodiment is collected as follows:

[0096] Design a simulated solar illumination in the simulation platform, and simulate the morning and evening changes of a day by defining the offset angle of the solar parameters relative to the scene world coordinates to simulate the morning and evening changes of a day: daytime refers to where when it represents direct sunlight, that is, 12 noon; night refers to and represents 12 midnight.

[0097] Define the following parameters and ranges to simulate various weathers in the real scene: cloud cover C (0% to 90%), rainfall R (0% to 80%), humidity W (0% to 100%), accumulated water volume P (0% to 85%), wind speed F (5% to 90%), snowfall S (0% to 80%), visibility O (0% to 30%).

[0098] Therefore, the weather data collected by the simulation platform is where the weather at the i-th timestamp is expressed as which can represent the complexity of the traffic environment under different lighting scenarios and different weather conditions.

[0099] Furthermore, the dataset augmentation in this embodiment refers to fusing the real dataset and the simulation dataset to create a more comprehensive and diverse dataset. To reflect randomness, the real dataset and the simulation dataset are formed into a synthetic training dataset according to the mixing ratio α, which can ensure that the model contacts both real data and synthetic data during the training process, so as to learn a more comprehensive data distribution and feature representation.

[0100] In a specific optional implementation, the twin traffic data and the real traffic data are formed into a mixed dataset including twins and reality according to the mixing ratio α, and then multi-time dimension fusion is performed to obtain the first fusion features including:

[0101] Use the following formula to represent the traffic data features in different time dimensions:

[0102] ;

[0103] ;

[0104] ;

[0105] where respectively represent the number of timestamps selected from hours, days, and weeks, and predictions are made in three dimensions at intervals of h timestamps, d timestamps, and w timestamps;

[0106] Normalize the traffic data features in three dimensions to obtain a normalized feature matrix ;

[0107] Adopt a feature pyramid network to fuse the normalized feature matrix to obtain a first fused feature.

[0108] Specifically, considering that traffic flow is periodic, not only the traffic flow in the previous few moments affects the current moment's flow, but also the traffic flow at the same moment in the previous few days and previous few weeks is relevant. Therefore, combine the traffic flow data in the previous few moments , previous few days and previous few weeks in three dimensions to predict the future traffic flow, which is expressed as follows:

[0109] ;

[0110] ;

[0111] ;

[0112] where respectively represent the number of timestamps selected from hours, days, and weeks, that is, predictions are made in three dimensions every h timestamps, d timestamps, and w timestamps.

[0113] Perform a normalization operation on the above three-dimensional data, and the formula is as follows:

[0114] ;

[0115] where is the normalized traffic flow data, X is the original data, , is the minimum and maximum values in X, and the normalized feature matrix is obtained.

[0116] In a specific optional embodiment, the feature fusion layer of the feature pyramid network has three layers, as shown in Figure 3 , Figure 3 which is a schematic diagram of using the feature pyramid network for feature fusion in this embodiment. Adopt the feature pyramid network to fuse the normalized feature matrix to obtain the first fused feature including:

[0117] By setting different time extraction windows, pass the normalized feature matrix through the extraction layer ResNet network of the feature pyramid network respectively to obtain the moment feature map , the daily feature map , the period feature map , the feature map extraction formula is as follows:

[0118] ;

[0119] where represents the feature map extracted from the ResNet network at the L-th layer, and L is 0, 1, 2 is the ResNet feature map extraction function is the weight matrix of each layer;

[0120] Take the moment feature map , the daily feature map , the period feature map as the layer, layer and layer of the feature pyramid network respectively;

[0121] Design three layers as the feature fusion layer, and its fusion formula is as follows:

[0122] ;

[0123] where represents the fusion result of the previous layer is , is the feature map of the extraction layer corresponding to the current fusion layer is the Hadamard product operation;

[0124] Convert the obtained fusion result into a feature vector through the global average pooling layer to retain the existing feature information in the feature map, and obtain the first fusion feature.

[0125] It should be noted that placing the moment feature map at the bottom layer can capture the changes and trends of traffic within a short time range; placing the daily feature map in the middle layer can capture the periodic patterns within a longer time range; placing the weekly feature map at the top layer can capture the seasonal or trend patterns within an even longer time range.

[0126] Furthermore, the final result obtained by the fusion layer , is converted into a feature vector through a global average pooling layer to retain the existing feature information in the feature map, and its formula is as follows:

[0127] ;

[0128] where is the global average pooling function, expressed as , where H, W, and C are the height, width, and number of channels of the input feature map, respectively. represents the feature value at position (i, j) in channel C of the input feature map.

[0129] S203: Feature extraction is performed based on weather data and traffic light information respectively to obtain weather features and traffic information features.

[0130] In this embodiment, the traffic signal status data S of the target road network includes three types of status information: red light, green light, and yellow light. According to historical experience, the red light status has an inhibitory effect on traffic flow, the green light status has a promoting effect on traffic flow, and the yellow light status has a relatively small impact on traffic flow. Therefore, the three types of data included in S are converted into three types of weights: , which are the weights assigned to the red, green, and yellow statuses respectively, and the weights decrease in sequence. For the traffic flow feature matrix within the same timestamp, perform feature product cross with the traffic light signal status data, that is .

[0131] Furthermore, after normalizing the traffic light duration data and performing splicing, then using the 1D convolution fusion method to complete the final fusion feature matrix , and the fusion formula is:

[0132] ;

[0133] where is the splicing function, is a one-dimensional convolutional layer to obtain the interaction relationship between the two data sources.

[0134] S204: Feature fusion is performed on the first fusion feature, weather feature, and traffic information feature to obtain the second fusion feature.

[0135] In a specific optional implementation manner, performing feature fusion on the first fusion feature, weather feature, and traffic information feature to obtain the second fusion feature includes:

[0136] Pass the weather feature matrix through the above ResNet network to obtain the feature map .

[0137] For use the ROIAlign method to perform weather feature extraction to obtain ;

[0138] Calculate the attention matrix between and the traffic flow feature map in the feature fusion layer of the feature pyramid , the attention feature map based on traffic characteristics , and the calculation formula is as follows:

[0139] ;

[0140] ;

[0141] where represents the feature map input to the i-th fusion layer, i = 0, 1, 2, and C represents the feature dimension of the current self-attention fusion;

[0142] Add the attention feature map based on traffic characteristics to the original to obtain the upper-layer fusion feature ;

[0143] Use the upper-layer fusion feature F as the input feature of the next-layer feature fusion layer of the feature pyramid for iterative fusion.

[0144] S205: Perform traffic prediction based on the second fusion feature to obtain a traffic prediction result.

[0145] In a specific optional implementation manner, performing traffic prediction based on the second fusion feature to obtain a traffic prediction result includes:

[0146] Use the adjacency matrix and the second fusion feature as the input of the generator in the generative adversarial network, and judge the authenticity of the prediction result based on the discriminator in the generative adversarial network;

[0147] When the discriminator determines that the prediction result is true, obtain the traffic prediction result.

[0148] Specifically, the prediction module of this embodiment uses a generative adversarial network model based on a spatio-temporal feature extraction unit, and this prediction model is beneficial to improving the generalization of the data captured by the above digital twin.

[0149] Further, the spatio-temporal feature extraction unit is composed of multiple layers of graph convolutional GCN, gated unit GRU, and multi-view graph attention layer. The graph convolutional GCN accepts the original node feature matrix and the fused adjacency matrix A as inputs, and generates a new low-dimensional feature representation for each node through the aggregation and information propagation of neighbor node features. The propagation formula for each node in the second layer is:

[0150] ;

[0151] where , is the self-connection matrix, is the degree matrix, is the weight matrix of the first layer and the second layer.

[0152] Furthermore, as a variant of the recurrent neural network (RNN), the gated unit GRU has the ability to process time series data. This layer captures and manages long-term dependencies by using memory units, update gates, and forget gates, effectively controlling the transmission and retention of information. The formulas for its components are as follows:

[0153] ;

[0154] where is the weight matrix corresponding to the gate component, , are activation functions.

[0155] In this embodiment, the overall architecture of the generative adversarial model consists of a generator (G) and a discriminator (D). The traffic map network is represented as a set of (V, E), where represents the set of nodes, that is, the traffic nodes in the network; represents the set of edges, that is, the connections or paths between nodes. Therefore, K = (V, E) is used to represent the topological structure of the traffic network. The prediction model is designed as follows:

[0156] Step 1: Design the distance adjacency matrix to represent the spatial proximity of two roads in the traffic map network K. The calculation formula is:

[0157] ;

[0158] where c represents the Euclidean distance between two nodes .

[0159] Calculate the similarity adjacency matrix to represent the traffic similarity of different roads. The calculation formula is:

[0160] ;

[0161] where represents the normalized feature vector of node i at time T, represents the normalized feature vector of node j at time T, softmax and ReLU are activation functions, is a learnable weight matrix. After normalizing the above two adjacency matrices, the final fused adjacency matrix is obtained, where , is the normalized adjacency matrix.

[0162] Step 2: Take the feature matrix and the fusion matrix A as the input of the generator. The loss function of the generator is as follows:

[0163] ;

[0164] where is the distribution function of the synthetic traffic data, which is unknown during the training process and needs to continuously adjust the parameters through adversarial training to minimize the gap with the true distribution. represents the generated data.

[0165] This loss function encourages the generator to generate samples that can be judged as real samples by the discriminator with a higher probability by maximizing the log probability that the generated samples are judged as real samples.

[0166] The input of the discriminator is real and fake samples and the distance matrix . The real samples are composed of real historical traffic flow data, and the fake samples are composed of the concatenation of real traffic historical flow data and future traffic historical flow data synthesized by the generator. The loss function of the discriminator is as follows:

[0167] ;

[0168] where S represents the real data, is the real data distribution function, which is obtained by analyzing the sampled real data.

[0169] The goal of this loss function is to enable the discriminator to accurately distinguish between the generated samples and the real samples. Therefore, this function is set to maximize the probability of judging the real samples as real and the probability of judging the generated samples as fake.

[0170] Step 3, for the graph convolutional generative adversarial network model, design an overall loss function that comprehensively considers MAE and RMSE. The MAE function is used to improve the prediction accuracy of small areas, and the RMSE function can be used to capture the traffic changes and fluctuations in the entire urban area. The calculation formula is as follows:

[0171] ;

[0172] where represent the predicted traffic flow data and the real traffic flow data respectively.

[0173] The overall loss function of the model is obtained by comprehensively considering the MAE loss and the RMSE loss and performing weighted summation. The calculation formula is as follows:

[0174] ;

[0175] By designing the value of the parameter , it is possible to comprehensively consider MAE and RMSE in traffic prediction to provide a more comprehensive evaluation, balance outliers and overall fluctuations, and make personalized trade-off choices according to task requirements.

[0176] Furthermore, the goal of the generative adversarial model is to achieve adversarial training of the generator and discriminator by minimizing the generator loss and maximizing the discriminator loss. The generator attempts to generate traffic samples similar to the real traffic distribution to reduce the probability that the discriminator identifies them as generated samples. The discriminator, on the other hand, attempts to accurately distinguish between real traffic and generated samples to increase the probability that the discriminator identifies the generated samples as such. Finally, the mean absolute error and RSME error are combined to adjust the model's requirement for the difference between generated data and real data. The parameter adjustment function of this model is as follows:

[0177] ;

[0178] where are the parameters of the generator and discriminator respectively.

[0179] In this embodiment, a digital twin simulation platform is used to construct a real urban traffic road scenario and collect twin traffic data; the twin traffic data and real traffic data are formed into a mixed dataset containing twins and reality in a mixing ratio of α, and then multi-time dimension fusion is performed to obtain the first fusion feature; feature extraction is performed based on weather data and traffic light information respectively to obtain weather features and traffic information features; feature fusion is performed on the first fusion feature, weather features and traffic information features to obtain the second fusion feature; traffic flow prediction is performed according to the second fusion feature to obtain the traffic flow prediction result. It realizes multi-level fusion processing of multi-dimensional traffic data, and further fuses with weather data and traffic signal data to obtain multi-dimensional feature data, improving the accuracy of traffic flow prediction in complex environments.

[0180] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution is prior or posterior. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0181] Figure 4 The principle block diagram of a traffic flow prediction device corresponding one-to-one to the traffic flow prediction method in the above embodiment is shown. As Figure 4 shown, the traffic flow prediction device includes a data acquisition module 31, a first fusion module 32, a feature extraction module 33, a second fusion module 34 and a traffic flow prediction module 35. The detailed description of each functional module is as follows:

[0182] The data acquisition module 31 is used to construct a real urban traffic road scenario by using a digital twin simulation platform and collect twin traffic data;

[0183] The first fusion module 32 is used to form a hybrid dataset containing twins and reality with a mixing ratio α for the twin traffic data and the real traffic data, and then perform multi-time dimension fusion to obtain the first fusion feature;

[0184] The feature extraction module 33 is used to extract features based on weather data and traffic light information respectively to obtain weather features and traffic information features;

[0185] The second fusion module 34 is used to perform feature fusion on the first fusion feature, the weather feature, and the traffic information feature to obtain the second fusion feature;

[0186] The traffic flow prediction module 35 is used to perform traffic flow prediction based on the second fusion feature to obtain a traffic flow prediction result.

[0187] Optionally, the first fusion module 32 includes:

[0188] The data representation unit is used to represent the traffic data features of different time dimensions by the following formula:

[0189] ;

[0190] ;

[0191] ;

[0192] Wherein, respectively represent the number of timestamps selected from hours, days, and weeks, and predictions are made in three dimensions at intervals of h timestamps, d timestamps, and w timestamps;

[0193] The normalization unit is used to normalize the traffic data features in three dimensions to obtain a normalized feature matrix ;

[0194] The fusion unit is used to use a feature pyramid network to fuse the normalized feature matrix to obtain the first fusion feature.

[0195] For the specific limitations of the traffic flow prediction device, reference may be made to the limitations of the traffic flow prediction method in the above text, which will not be elaborated here. Each module in the above traffic flow prediction device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0196] To solve the above technical problems, an embodiment of the present application also provides a computer device. For details, please refer to Figure 5 ,Figure 5 This is the basic structural block diagram of the computer device in this embodiment.

[0197] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are communicatively connected to each other via a system bus. It should be noted that only the computer device 4 with components connected to the memory 41, the processor 42, and the network interface 43 is shown in the figure. However, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Among them, those skilled in the art of this technology can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0198] The computer device can be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The computer device can perform human-computer interaction with the user through a keyboard, a mouse, a remote control, a touchpad, or a voice control device, etc.

[0199] The memory 41 includes at least one type of readable storage medium. The readable storage medium includes flash memory, hard disks, multimedia cards, card-type memories (such as SD or D interface display memories, etc.), random access memories (RAMs), static random access memories (SRAMs), read-only memories (ROMs), electrically erasable programmable read-only memories (EEPROMs), programmable read-only memories (PROMs), magnetic memories, magnetic disks, optical disks, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as the hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device 4. Of course, the memory 41 can also include both the internal storage unit and the external storage device of the computer device 4. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as the program code of the traffic flow prediction method. In addition, the memory 41 can also be used to temporarily store various data that have been output or will be output.

[0200] In some embodiments, the processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run the program code stored in the memory 41 or process data, such as running the program code of the traffic flow prediction method.

[0201] The network interface 43 may include a wireless network interface or a wired network interface, and this network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0202] The present application also provides another implementation manner, that is, to provide a computer-readable storage medium storing an interface display program, and the interface display program can be executed by at least one processor to enable the at least one processor to execute the steps of the traffic flow prediction method as described above.

[0203] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation manner. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc), and includes several instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in the various embodiments of the present application.

[0204] Obviously, the above-described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The drawings show the preferred embodiments of the present application, but do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure content of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the foregoing embodiments, for those skilled in the art, they can still modify the technical solutions described in the foregoing specific embodiments, or perform equivalent replacements on some of the technical features. Any equivalent structure made by using the specification and drawings of the present application, directly or indirectly applied in other related technical fields, is similarly within the scope of the patent protection of the present application.

Claims

1. A traffic flow prediction method, characterized in that Including: Construct a real urban traffic road scene using a digital twin simulation platform and collect twin traffic data; Form a hybrid dataset containing twins and real data with a mixing ratio α for the twin traffic data and the real traffic data, and then perform multi-time dimension fusion to obtain the first fusion feature. Use the multi-source data fusion module to perform multi-level fusion on the twin traffic data and the real traffic data generated by the traffic digital twin platform. According to the simulation vehicle position data of the above traffic digital twin platform, calculate the number of vehicles in each intersection area within each time interval to obtain the traffic flow data , obtain the current state duration feature data of the simulated traffic lights = , ... , the traffic signal status data S of the target intersection and the scene weather parameter data , obtain the urban intersection traffic flow data obtained by real urban sensors and use the traffic flow data to perform dataset augmentation, and finally obtain the final fusion feature matrix through the multi-source fusion model ; Extract features based on weather data and traffic light information respectively to obtain weather features and traffic information features; Perform feature fusion on the first fusion feature, the weather feature and the traffic information feature to obtain a second fusion feature; Perform traffic flow prediction based on the second fusion feature to obtain a traffic flow prediction result; Among them, the collection of twin traffic data includes: Through the API interface of Carla, collect the status and duration of traffic lights and the weather data of the current simulation scene in real time; Utilize the scene generation ability provided by Carla to generate ordinary scenes and target specific scenes based on different urban traffic environments by directionally adjusting traffic flow, weather conditions, and signal timing parameters. The target specific scenes include simulated harsh environment scenes, urban morning rush hour scenes, and urban road accident scenes; Based on various scenes, use the built-in sensors of the Carla simulation platform to collect traffic data. Simulate the real traffic scene by changing the number and type of vehicles, weather, and traffic light configuration in the scene, and use the point cloud data obtained by the RGB camera and lidar sensors to record the trajectory data of vehicles in the scene, including the position coordinates, speed, and deflection angle of vehicles within each timestamp, to obtain the twin traffic data.

2. The traffic flow prediction method according to claim 1, wherein The formation of a hybrid dataset containing twins and reality with a mixing ratio α for the twin traffic data and real traffic data, and then performing multi-time dimension fusion to obtain the first fusion feature includes: Use the following formula to represent the traffic data features in different time dimensions: Among them, , , respectively represent the number of timestamps selected from hours, days, and weeks, and predictions are made in three dimensions at intervals of h timestamps, d timestamps, and w timestamps; Perform data normalization on the traffic data features in three dimensions to obtain a normalized feature matrix ; Using a feature pyramid network, fuse the normalized feature matrix to obtain the first fused feature.

3. The traffic flow prediction method according to claim 2, wherein The feature pyramid network is adopted to fuse the normalized feature matrix to obtain the first fused feature, including: By setting different time extraction windows, the normalized feature matrix is respectively passed through the extraction layer ResNet network of the feature pyramid network to obtain the moment feature map , the day feature map , the cycle feature map , and the feature map extraction formula is as follows: Among them represents the feature map extracted by the ResNet network at the L-th layer, where L is 0, 1, 2, is the function for the ResNet to extract the feature map, are the weight matrices of each layer; Take the moment feature map , the day feature map , and the cycle feature map as the layer, layer, and layer of the feature pyramid network respectively; Design , , The third layer serves as a feature fusion layer, and its fusion formula is as follows: Among them represents the fusion result of the upper layer, is , which is the feature map of the extraction layer corresponding to the current fusion layer, is the Hadamard product operation; Convert the obtained fusion result into a feature vector through a global average pooling layer to retain the existing feature information in the feature map and obtain the first fusion feature.

4. The traffic flow prediction method according to claim 1, characterized in that The feature fusion of the first fusion feature, the weather feature and the traffic information feature to obtain the second fusion feature includes: The scene weather parameter data Obtain a feature map through the ResNet network ; Pairwise Use the ROIAlign method to extract weather features and obtain ; Calculation The attention matrix between the flow feature map in the feature fusion layer of the feature pyramid , the attention feature map based on the flow feature , the calculation formula is as follows: Among them represents the feature map input to the i-th fusion layer, where i = 0, 1, 2, and C represents the feature dimension of the current self-attention fusion; Add the attention feature map based on traffic characteristics to and obtain the fused feature F of the upper layer = + ; Use the upper layer fusion feature F as the input feature of the next layer feature fusion layer of the feature pyramid for iterative fusion.

5. The traffic flow prediction method according to claim 1, wherein The traffic flow prediction based on the second fusion feature to obtain the traffic flow prediction result includes: Use the adjacency matrix and the second fusion feature as the input of the generator in the generative adversarial network, and judge the authenticity of the prediction result based on the discriminator in the generative adversarial network; When the discriminator determines that the prediction result is true, obtain the traffic flow prediction result.

6. A traffic flow prediction device, characterized in that, Including: A data collection module for constructing a real urban traffic road scene using a digital twin simulation platform and collecting twin traffic data; The first fusion module is used to form a hybrid dataset containing twins and reality with a mixing ratio α for the twin traffic data and the real traffic data, and then perform multi-time dimension fusion to obtain the first fusion feature. The multi-source data fusion module is adopted to perform multi-level fusion on the twin traffic data and the real traffic data generated by the traffic digital twin platform. According to the simulation vehicle position data of the above traffic digital twin platform, the vehicle flow data is calculated by obtaining the number of vehicles in each intersection area within each time interval , and obtain the current state duration feature data of the simulation traffic lights = , ... , the traffic signal state data S of the target intersection and the scene weather parameter data , obtain the urban intersection vehicle flow data obtained by the real urban sensors and use for dataset augmentation, and finally obtain the final fusion feature matrix through the multi-source fusion model ; A feature extraction module for extracting features based on weather data and traffic light information respectively to obtain weather features and traffic information features; A second fusion module for performing feature fusion on the first fusion feature, the weather feature and the traffic information feature to obtain a second fusion feature; A traffic flow prediction module for performing traffic flow prediction based on the second fusion feature to obtain a traffic flow prediction result; The collection of twin traffic data includes: Through the API interface of Carla, the status and duration of traffic lights and the weather data of the current simulation scenario are collected in real time; Using the scenario generation ability provided by Carla, by directionally adjusting parameters such as traffic flow, weather conditions, and signal timing, ordinary scenarios and target-specific scenarios based on different urban traffic environments are generated, and the target-specific scenarios include simulated harsh environment scenarios, urban morning rush hour scenarios, and urban road accident scenarios; Based on various scenarios, use the sensors built into the Carla simulation platform to collect traffic data. Simulate the real traffic scenario by changing the number and type of vehicles, weather, and traffic light configuration in the scenario. Use the point cloud data obtained by the RGB camera and lidar sensor to record the trajectory data of vehicles in the scenario, including the position coordinates, speed, and deflection angle of the vehicle at each timestamp, to obtain the twin traffic data.

7. The traffic flow prediction device according to claim 6, wherein The first fusion module includes: A data characterization unit for representing the traffic data characteristics in different time dimensions using the following formula: Among them, , , respectively represent the number of timestamps selected from hours, days, and weeks, and predictions are made in three dimensions of every h timestamps, d timestamps, and w timestamps; A normalization unit, configured to perform data normalization on traffic data features in three dimensions to obtain a normalized feature matrix ; A fusion unit, which is used to adopt a feature pyramid network to fuse the normalized feature matrix to obtain a first fused feature.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the traffic flow prediction method according to any one of claims 1 to 5.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the traffic flow prediction method according to any one of claims 1 to 5.

Images