Automatic driving simulation test background traffic flow generation method based on diffusion generation
By using a traffic flow generation method based on diffusion generation and a traffic flow generation model with vectorized representation and efficient attention mechanism, the problems of high computational complexity and long inference time in existing technologies are solved, and high-fidelity traffic flow generation is achieved efficiently and quickly, meeting the complex environment requirements of autonomous driving simulation testing.
Patent Information
- Application Number
- CN202510811099.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-17
- Publication Date
- 2025-09-16
AI Technical Summary
Existing simulation traffic flow generation methods have high computational complexity, long inference time and cannot meet the complexity requirements of the real world. Existing methods cannot effectively evaluate the decision-making and response of autonomous driving algorithms in complex traffic scenarios.
A traffic flow generation method based on diffusion generation is adopted. By vectorizing the real scene data into a rasterized state image, and using the encoder, pre-trained DIT module and decoder to build a traffic flow generation model, combined with an efficient attention mechanism and flow matching method, future traffic scenes and traffic agent behaviors are generated.
It achieves efficient and rapid generation of high-fidelity traffic flows, reduces computational complexity and inference time, improves the robustness and real-time response capabilities of the model, and can effectively evaluate the behavior of autonomous vehicles in complex environments.
Smart Images
Figure CN120654574A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of simulated traffic flow generation, and specifically relates to a method for generating background traffic flow for autonomous driving simulation tests based on diffusion generation. Background Art
[0002] Traffic flow simulation technology plays a crucial role in evaluating the safety of autonomous driving algorithms' decisions in complex traffic scenarios and the rationality of their responses to other traffic participants. Existing technical solutions in the field of traffic simulation and flow generation fall into three main categories: first, using rule-based simulation platforms to generate traffic flows for specific scenarios; second, building platforms and replaying real traffic trajectories; and third, using data-driven generative models to generate future traffic trajectories. However, existing technologies still have the following drawbacks: existing rule-based traffic flow generation methods manually create traffic generation rules and corresponding map libraries based on experience. However, due to cognitive limitations and the variability of scene environments, the diversity of test scenarios is severely insufficient and far from meeting the complexity of the real world. This makes it difficult to evaluate the key decisions made by autonomous driving algorithms in complex traffic scenarios and their responses to other traffic participants. While replaying real traffic trajectories maintains the fidelity of real-world scenarios, collecting large-scale scenario data is time-consuming and expensive; commonly used open-source driving datasets often suffer from fragmented and incomplete trajectories; and users have limited controllability and interaction with scenarios and trajectories, making them unable to fully simulate the behavior of autonomous vehicles in complex environments. Existing data-driven generative models first generate traffic flows based on vectorized representations of different traffic attributes in real-world scenarios. This irregular input format often makes model learning difficult. Secondly, to improve generation quality, existing generative methods use a global attention mechanism to extract feature information and employ multi-step denoising to iteratively generate the target result. This significantly increases computational complexity and inference time, making it impossible to meet the demand for real-time response to traffic flow generation in simulation environments.
[0003] In summary, the existing simulation traffic flow generation methods still have problems such as high computational complexity, long inference time and inability to meet the complexity of the real world. Therefore, it is very necessary to propose a new method to solve the above problems. Summary of the Invention
[0004] The purpose of this invention is to solve the problems of high computational complexity, long inference time and inability to meet the complexity of the real world in existing simulation traffic flow generation methods, and to propose a background traffic flow generation method for autonomous driving simulation test based on diffusion generation.
[0005] The technical solution adopted by the present invention to solve the above technical problems is: a method for generating background traffic flow for autonomous driving simulation testing based on diffusion generation, the method specifically comprising the following steps:
[0006] Step 1: Obtain a data set consisting of data from various real scenes, then represent the data from each real scene using vectors, and map the vectors corresponding to each real scene to generate a rasterized state image of each real scene;
[0007] Step 2: Build a traffic flow generation model, which includes an encoder, a pre-trained DIT module, and a decoder;
[0008] In the traffic flow generation model, the input of the traffic flow generation model is used as the input of the encoder. The output of the encoder and the one-hot encoding result of the label of the city where the real scene is located are used as the input of the pre-trained DIT module. The output of the pre-trained DIT module is used as the input of the decoder, and the output of the decoder is used as the output of the traffic flow generation model.
[0009] Step 3: The rasterized state images of each real scene are used as inputs of the traffic flow generation model. The traffic flow generation model simulates and generates traffic flow based on the input rasterized state images.
[0010] Furthermore, the vector representation of the real scene data is ;
[0011] in, Represents lane geometry;
[0012] A is the connection relationship matrix between adjacent roads;
[0013] Represents the spatiotemporal state of pedestrians;
[0014] Represents the spatiotemporal state of the vehicle;
[0015] Represents the spatiotemporal state of a static object;
[0016] Indicates the speed of the vehicle.
[0017] Furthermore, the encoder is ResNet50.
[0018] Furthermore, the attention mechanism in the pre-trained DIT module is an efficient attention mechanism.
[0019] Furthermore, the working process of the efficient attention mechanism is as follows:
[0020] Step 1: Combine the input and weight of the efficient attention mechanism Perform matrix multiplication to obtain the query vector Q; combine the input of the efficient attention mechanism with the weight Perform matrix multiplication to obtain the key vector K; combine the input of the efficient attention mechanism with the weight Perform matrix multiplication to obtain the value vector V;
[0021] Step 2: Perform a pooling operation on the query vector Q to obtain the proxy vector A;
[0022] Step 3: Treat the proxy vector A as a query, and calculate the proxy feature by adding the first Softmax attention layer with proxy bias to A, K, and V. ; Then the proxy feature As the value vector, the proxy vector A is used as the key vector, and A is combined with , Q is passed through the second Softmax attention layer with added proxy bias, and the output of the second Softmax attention layer is obtained:
[0023]
[0024] Among them, the superscript T represents transposition;
[0025] represents the proxy bias of the first Softmax attention layer;
[0026] represents the proxy bias of the second Softmax attention layer;
[0027] Represents the Softmax function;
[0028] Represents the output of the second Softmax attention layer;
[0029] Step 4: Pass the value vector V through the deep convolution unit, and then add the output of the deep convolution unit to the output of the second Softmax attention layer to obtain the output of the efficient attention mechanism. :
[0030]
[0031] in, Represents a depthwise convolutional unit.
[0032] Furthermore, the loss function used in the pre-training process of the DIT module is for:
[0033]
[0034] in, A latent encoding vector representing the true data distribution;
[0035] represents a latent code vector randomly sampled from a standard normal distribution;
[0036] represents the 2-norm;
[0037] represents the predicted velocity field, Equal to The latent code vector of time steps Output from the DiT module ;
[0038] Expressing expectation.
[0039] Furthermore, the The latent code vector of time steps for:
[0040] .
[0041] Furthermore, the decoder is a decoder of the Transformer architecture.
[0042] Furthermore, the traffic flow generated by the traffic flow generation model simulation specifically includes lane geometry features, red light traffic flow information, green light traffic flow information, the spatiotemporal state of vehicles, the spatiotemporal state of pedestrians, and the spatiotemporal state of static objects.
[0043] The beneficial effects of the present invention are:
[0044] The present invention converts the input information of historical real scenes into vectorized representations, and then converts the vectorized representations into a unified rasterized scene representation. The rasterized scene representation simultaneously retains the road topology information and the interactive relationship between dynamic traffic entities. The rasterized scene representation is input into a traffic flow generation model based on a diffusion generation method based on flow matching and an efficient attention mechanism. The efficient attention mechanism achieves an elegant integration of the advantages of softmax attention and linear attention by introducing the concept of proxy tokens, which retains high expressiveness and linear complexity. The flow matching method is used to replace the traditional denoising diffusion probability model, making the sampling path smoother during inference and reducing the inference time. The traffic flow generation model can effectively learn various traffic attributes from real scene data sets. The traffic flow generation model can simulate and generate future traffic scenes and the future behaviors and trajectories of all traffic agents in the traffic scenes. The traffic flow simulation generation method of the present invention has good robustness to the complex real world.
[0045] Extensive experimental validation of the present invention on the nuPlan dataset demonstrates that the proposed method outperforms existing state-of-the-art methods in terms of both generation efficiency and realism. While maintaining high-quality trajectory generation, it can reduce the consumption of computing resources, can be deployed locally in a lightweight manner, and can quickly generate high-fidelity traffic flows, demonstrating its broad development prospects. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 This is a flow chart of a method for generating background traffic flow for autonomous driving simulation testing based on diffusion generation according to the present invention;
[0047] Figure 2 This is a flowchart of an efficient attention mechanism. DETAILED DESCRIPTION
[0048] Specific implementation method 1: Combination Figure 1 This embodiment describes a method for generating background traffic flow for an autonomous driving simulation test based on diffusion generation, and the method specifically includes the following steps:
[0049] Step 1: Obtain a dataset consisting of data from various real-world scenarios. Represent each real-world scenario's data using vectors, and map the vectors corresponding to each real-world scenario to generate a rasterized state image (RSI) for each real-world scenario.
[0050] Step 2: Build a traffic flow generation model, which includes an encoder, a pre-trained DIT module, and a decoder;
[0051] In the traffic flow generation model, the input of the traffic flow generation model is used as the input of the encoder. The output of the encoder and the one-hot encoding result of the label of the city where the real scene is located (city labels include left and right steering) are used as the input of the pre-trained DIT module. The output of the pre-trained DIT module is used as the input of the decoder, and the output of the decoder is used as the output of the traffic flow generation model.
[0052] Step 3: The rasterized state images of each real scene are used as input to the traffic flow generation model. The traffic flow generation model simulates and generates traffic flow based on the input rasterized state images (the generated traffic flow is the output of the traffic flow generation model).
[0053] After the traffic flow generation model of the present invention predicts the future traffic flow state, it can also return to guide the control of the autonomous driving vehicle based on the predicted traffic flow state.
[0054] The present invention proposes a model framework that combines the traditional diffusion model training process, the diffusion generation method of flow matching, and the efficient attention mechanism. The model framework can quickly infer given real scene information and a one-hot city label control information, so that the agent in the scene generates realistic traffic behavior. Taking advantage of the linear computational complexity of the efficient attention mechanism, the computational complexity during training and inference can be effectively reduced, so that the model can be deployed in a lightweight manner and reduce dependence on high computing power. Introducing the modified flow algorithm in the traffic flow generation task can also avoid the high time waste caused by the multi-step denoising required during traditional DDPMs model inference, thereby meeting the demand for real-time response.
[0055] Specific embodiment 2: This embodiment differs from the specific embodiment 1 in that the vector representation of the real scene data is ;
[0056] in, Represents lane geometry (each lane is represented by a fixed set of 20 bird's-eye view (BEV) points, denoted as );
[0057] A is the connection relationship matrix between adjacent roads; the present invention encodes the endpoint sharing information between the current lane and the predecessor and successor lanes into A, which is recorded as ,When there is a connection relationship between two lanes, the corresponding elements of the two lanes in the connection relationship matrix are 1, otherwise they are 0. N is the number of lanes in a certain field of view (FOV);
[0058] Represents the spatiotemporal state of pedestrians;
[0059] Represents the spatiotemporal state of the vehicle;
[0060] Represents the spatiotemporal state of a static object;
[0061] Indicates the speed of the vehicle.
[0062] Other steps and parameters are the same as those in the first embodiment.
[0063] The present invention evaluates the decision-making safety and response of an autonomous vehicle in a real scenario. Therefore, a vector representation of the autonomous vehicle is constructed as a self-driving vehicle, and future traffic flow conditions are predicted based on the constructed vector representation.
[0064] Specific implementation method three: This implementation method is different from specific implementation methods one or two in that the encoder is ResNet50.
[0065] Other steps and parameters are the same as those in the first or second embodiment.
[0066] The feature vector output by the encoder consists of two separate parts and , Represents information related to the lanes in the scene, Indicates information related to the agent.
[0067] Specific embodiment four: This embodiment differs from any one of specific embodiments one to three in that the attention mechanism in the pre-trained DIT module is an efficient attention mechanism.
[0068] The other steps and parameters are the same as those in the first to third embodiments.
[0069] The present invention replaces the attention mechanism of the original DIT module with an efficient attention mechanism on the basis of the original DIT module, while the other structures of the DIT module remain unchanged.
[0070] Specific implementation method five: Combination Figure 2 This embodiment differs from the first to fourth embodiments in that the working process of the efficient attention mechanism is as follows:
[0071] Step 1: Combine the input and weight of the efficient attention mechanism Perform matrix multiplication to obtain the query vector Q; combine the input of the efficient attention mechanism with the weight Perform matrix multiplication to obtain the key vector K; combine the input of the efficient attention mechanism with the weight Perform matrix multiplication to obtain the value vector V;
[0072] Step 2: Pool the query vector Q to obtain the agent vector A (agent token). ;
[0073] The query vector Q has dimensions N×d, and the resulting proxy vector A has dimensions n×d. N is the number of tokens in the serialized image, d is the number of attention heads in the multi-head attention mechanism, and n is a hyperparameter. By controlling the hyperparameter n, we can make it much smaller than N. Therefore, the proxy vector A can be viewed as a compressed vector of the query vector Q. This ensures that the representation is as close to the query vector Q as possible while reducing the corresponding vector size and computational complexity.
[0074] Step 3: Treat the proxy vector A as a query, and calculate the proxy feature by adding the first Softmax attention layer with proxy bias to A, K, and V. , the agent vector A is used to efficiently aggregate global information from the key vector K and the value vector V (Agent Aggregation); then the agent feature As the value vector, the proxy vector A is used as the key vector, and A is combined with , Q is passed through the second Softmax attention layer with added proxy bias, and the output of the second Softmax attention layer is obtained:
[0075]
[0076] Among them, the superscript T represents transposition;
[0077] represents the proxy bias of the first Softmax attention layer, ;
[0078] represents the proxy bias of the second Softmax attention layer, ;
[0079] Represents the Softmax function;
[0080] Represents the output of the second Softmax attention layer;
[0081] Step 4: Pass the value vector V through the deep convolution unit (DWC), and then add the output of the deep convolution unit to the output of the second Softmax attention layer to obtain the output of the efficient attention mechanism. :
[0082]
[0083] in, Represents a depthwise convolutional unit.
[0084] The other steps and parameters are the same as those in the first to fourth embodiments.
[0085] The present invention completes the two-step information flow process of proxy broadcast by feeding the aggregated global context information back to the query vector Q. This approach not only maintains linear computational complexity, but also retains strong expressive power, thereby effectively reducing computational costs while ensuring the superior performance of the model. At the same time, in order to better utilize position information, the present invention also introduces a proxy bias to the Softmax attention layer, so that the Softmax attention layer can pay more attention to the required information during attention calculations, and maintains the feature representation capability of the module by adopting a deep convolution module (DWC) to retain feature diversity, so that the method of the present invention can improve the expressive power of the model while reducing computational complexity.
[0086] Specific embodiment 6: This embodiment differs from specific embodiments 1 to 5 in that the loss function used in the pre-training process of the DIT module is for:
[0087]
[0088] in, A latent encoding vector representing the true data distribution;
[0089] represents a latent code vector randomly sampled from a standard normal distribution;
[0090] represents the 2-norm;
[0091] represents the predicted velocity field, Equal to The latent code vector of time steps Output from the DiT module , Represents the network parameters of the DiT module;
[0092] Expressing expectation.
[0093] The other steps and parameters are the same as those in the first to fifth embodiments.
[0094] By minimizing the squared loss of the difference between the interpolated path speed and the true speed, the model learns a generative flow that approximates a straight line. This sampling process allows the path from the initial distribution to the target distribution to be linear or piecewise linear, greatly simplifying the denoising process and accelerating sampling. Compared with traditional DiT models, this method significantly reduces inference latency while maintaining sample quality. Compared with pure RF frameworks, this method leverages the powerful expressive power and parallel computing advantages of the Transformer to achieve high-fidelity traffic trajectory generation in a high-dimensional latent space.
[0095] Specific embodiment seven: This embodiment differs from the specific embodiments one to six in that the The latent code vector of time steps for:
[0096]
[0097] The other steps and parameters are the same as those in the first to sixth embodiments.
[0098] Specific embodiment eight: This embodiment differs from any one of specific embodiments one to seven in that the decoder is a decoder of the Transformer architecture.
[0099] The other steps and parameters are the same as those in the first to seventh embodiments.
[0100] The Transformer architecture's decoder maps the output of the diffusion model into a variable-sized set of entities. This architecture transforms irregular vectorized inputs into a unified, compact representation that supports information sharing, while constructing a fixed-dimensional diffusion latent space. Finally, the present invention also preserves structured representations of different scene attributes, facilitating subsequent simulation tasks.
[0101] Specific embodiment nine: This embodiment differs from any one of specific embodiments one to eight in that the traffic flow generated by the traffic flow generation model simulation specifically includes lane geometry features, red light traffic information (lane areas that are impassable due to red lights), green light traffic information (lane sections that are safe to pass under green light conditions), the spatiotemporal state of vehicles, the spatiotemporal state of pedestrians, and the spatiotemporal state of static objects.
[0102] The other steps and parameters are the same as those in Specific Embodiments 1 to 8.
[0103] The above examples are merely illustrative of the calculation model and process of the present invention and are not intended to limit the embodiments of the present invention. Persons skilled in the art will readily appreciate that other variations or modifications based on the above description are possible. This list of embodiments is not exhaustive; however, any obvious variations or modifications derived from the technical solution of the present invention remain within the scope of protection of the present invention.
Claims
1. A method for generating background traffic flow for autonomous driving simulation test based on diffusion generation, characterized in that: The method specifically comprises the following steps: Step 1: Obtain a data set consisting of data from various real scenes, then represent the data from each real scene using vectors, and map the vectors corresponding to each real scene to generate a rasterized state image of each real scene; Step 2: Build a traffic flow generation model, which includes an encoder, a pre-trained DIT module, and a decoder; In the traffic flow generation model, the input of the traffic flow generation model is used as the input of the encoder. The output of the encoder and the one-hot encoding result of the label of the city where the real scene is located are used as the input of the pre-trained DIT module. The output of the pre-trained DIT module is used as the input of the decoder, and the output of the decoder is used as the output of the traffic flow generation model. Step 3: The rasterized state images of each real scene are used as inputs of the traffic flow generation model. The traffic flow generation model simulates and generates traffic flow based on the input rasterized state images.
2. The method for generating background traffic flow for autonomous driving simulation test based on diffusion generation according to claim 1, characterized in that: The vector representation of the real scene data is ; in, Represents lane geometry; A is the connection relationship matrix between adjacent roads; Represents the spatiotemporal state of pedestrians; Represents the spatiotemporal state of the vehicle; Represents the spatiotemporal state of a static object; Indicates the speed of the vehicle.
3. The method for generating background traffic flow for autonomous driving simulation test based on diffusion generation according to claim 1, characterized in that: The encoder is ResNet50.
4. The method for generating background traffic flow for autonomous driving simulation test based on diffusion generation according to claim 1, characterized in that: The attention mechanism in the pre-trained DIT module is an efficient attention mechanism.
5. The method for generating background traffic flow for autonomous driving simulation test based on diffusion generation according to claim 4, characterized in that: The working process of the efficient attention mechanism is as follows: Step 1: Combine the input and weight of the efficient attention mechanism Perform matrix multiplication to obtain the query vector Q; combine the input of the efficient attention mechanism with the weight Perform matrix multiplication to obtain the key vector K; combine the input of the efficient attention mechanism with the weight Perform matrix multiplication to obtain the value vector V; Step 2: Perform a pooling operation on the query vector Q to obtain the proxy vector A; Step 3: Treat the proxy vector A as a query, and calculate the proxy feature by adding the first Softmax attention layer with proxy bias to A, K, and V. ; Then the proxy feature As the value vector, the proxy vector A is used as the key vector, and A is combined with , Q is passed through the second Softmax attention layer with added proxy bias, and the output of the second Softmax attention layer is obtained: Among them, the superscript T represents transposition; represents the proxy bias of the first Softmax attention layer; represents the proxy bias of the second Softmax attention layer; Represents the Softmax function; Represents the output of the second Softmax attention layer; Step 4: Pass the value vector V through the deep convolution unit, and then add the output of the deep convolution unit to the output of the second Softmax attention layer to obtain the output of the efficient attention mechanism. : in, Represents a depthwise convolutional unit.
6. The method for generating background traffic flow for autonomous driving simulation test based on diffusion generation according to claim 1, characterized in that: The loss function used in the pre-training process of the DIT module is for: in, A latent encoding vector representing the true data distribution; represents a latent code vector randomly sampled from a standard normal distribution; represents the 2-norm; represents the predicted velocity field, Equal to The latent code vector of time steps Output from the DiT module ; Expressing expectation.
7. The method for generating background traffic flow for autonomous driving simulation test based on diffusion generation according to claim 6, characterized in that: The said The latent code vector of time steps for: 。 8. The method for generating background traffic flow for autonomous driving simulation test based on diffusion generation according to claim 1, characterized in that: The decoder is a Transformer architecture decoder.
9. According to the diffusion-generated background traffic flow generation method for autonomous driving simulation testing according to claim 1, the traffic flow generated by the traffic flow generation model simulation specifically includes lane geometric characteristics, red light traffic flow information, green light traffic flow information, the spatiotemporal state of vehicles, the spatiotemporal state of pedestrians, and the spatiotemporal state of static objects.