Large-scale road simulation test scene library generation method

Abstract

The invention relates to a large-scale road simulation test scene library generation method, which comprises the following steps: acquiring real road structure data, and constructing a road structure data set; the method comprises the following steps: establishing a road structure generation model based on a Unet network architecture and a wide and deep network architecture of FreeU; training a road structure generation model by using the road structure data set to obtain a road structure diffusion generation model; inputting the road attributes and the random noise into a road structure diffusion generation model, and outputting to obtain road scene data; and evaluating, sorting and screening the output road scene data, and constructing a road scene library for an automatic driving vehicle simulation test. Compared with the prior art, the method can effectively generate a real and diversified comprehensive road scene library for testing the automatic driving vehicle.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving test technology, and in particular to a method for generating a large-scale road simulation test scenario library. Background Art

[0002] Testing and validating autonomous vehicles requires a large-scale scenario library, and road scenarios are the basis of autonomous driving testing. However, there is a serious lack of comprehensive road scenario libraries suitable for digital twin-based simulation testing. The complexity and variability of real-world road scenarios seriously affect the safety and overall performance of autonomous vehicles. A key bottleneck in autonomous vehicle simulation testing is that the various road scenarios used for simulation testing purposes are limited, and the existing scenarios are single in type and simple in structure. Therefore, there is an urgent need for an effective road scenario generation method to generate realistic and diverse road scenarios to enhance autonomous vehicle testing.

[0003] Autonomous vehicles operate in fundamentally different ways than vehicles driven by humans. However, the existing road infrastructure is mainly designed for human drivers and cannot adapt well to the operation of autonomous vehicles. To ensure seamless integration and optimize performance, the road infrastructure needs to be adjusted accordingly. In other words, there is an urgent need for a comprehensive road scenario library tailored for digital twin-based autonomous driving testing. Such a road scenario library is crucial to guiding the design of future road infrastructure, ensuring that it meets the unique requirements of autonomous vehicles and maximizes its potential benefits.

[0004] In order to achieve high-precision autonomous driving simulation testing, relevant researchers have proposed a variety of road scene formats, including OpenDrive, Lanelet2, and CommonRoad. Among them, OpenDrive is widely used by most well-known companies and simulation test platforms, such as Intel's CARLA, VTD (Virtual Test Drive), Apollo, PreScan, and SUMO. Although road scene simulation technology is widely used, there is little research on generating realistic and diverse road scenes. The current methods for constructing test road scenes mainly rely on aerial images, sensor data reconstruction, or manual creation using commercial tools such as MATLAB RoadRunner. In addition, the CommonRoad scenario designer is designed to convert OpenStreetMap maps to CommonRoad format. This scenario designer can only realize scene format conversion and cannot generate more scenarios for autonomous driving vehicle testing.

[0005] In addition, the researchers proposed a model-driven approach to construct a topology graph to generate a variety of overpass scenarios. However, this approach can only generate specific types of overpass scenarios.

[0006] In summary, existing technologies are limited to converting road scene formats or generating a single type of road scene based on existing data, and are unable to generate large-scale, realistic and diverse road scenes. Summary of the invention

[0007] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a large-scale road simulation test scenario library generation method, which can effectively generate a real and diverse comprehensive road scenario library for autonomous driving vehicle testing.

[0008] The purpose of the present invention can be achieved by the following technical solution: A method for generating a large-scale road simulation test scenario library comprises the following steps:

[0009] S1. Collect real road structure data and construct a road structure data set;

[0010] S2. Establish a road structure generation model based on FreeU's Unet network architecture and wide-deep network architecture;

[0011] S3, using the road structure data set to train the road structure generation model to obtain a road structure diffusion generation model;

[0012] S4, inputting road attributes and random noise into a road structure diffusion generation model, and outputting road scene data;

[0013] S5. Evaluate, sort and filter the output road scene data to build a road scene library for autonomous driving vehicle simulation testing.

[0014] Furthermore, the step S1 includes the following process:

[0015] S11. Collecting real road structure data by web crawling;

[0016] S12. For the collected real road structure data, the coordinate data of the corresponding road shape control points are extracted, and the interpolation method is used to resample the data to a fixed length to construct a road structure data set.

[0017] Furthermore, in the road structure generation model established in step S2, the Unet network architecture is used to predict the noise of each diffusion time step, and the wide-deep network architecture is used to encode and embed road attribute information.

[0018] Furthermore, the Unet network architecture includes a plurality of Road-UNet networks connected in sequence, each Road-UNet network includes a down-sampling layer and an up-sampling layer, the down-sampling layer and the up-sampling layer each include a plurality of stacked Resnet (ResidualNetwork) modules based on 1D-CNN (1D Convolutional Neural Network), a transition module based on an attention mechanism is integrated between the plurality of Resnet modules, and the input end of the Resnet module is connected to two fully connected layers with shared parameters.

[0019] Furthermore, the Unet network architecture introduces multi-head FreeU operations to balance the mapping of backbone features and jump features in the neural network, where Road-UNet contains two scalar factors: z The backbone feature scaling factor b z and for h z The jump feature scaling factor s z , where x z represents the backbone features in the zth block in the Road-UNet decoder, h z represents the skip connection feature in the zth block in the Road-UNet decoder. For the backbone feature x z , according to the average feature map Adaptively adjust the scaling factor α z :

[0020]

[0021] Among them, b z is a scalar constant, Represents the feature map x z The jth channel in, B represents x z The total number of channels in , ⊙ represents the element-wise product;

[0022] For the frequency hopping characteristic h z , using spectrum modulation in the Fourier domain to selectively attenuate the low-frequency components of the frequency hopping feature:

[0023] h′ z,j =IFFT(FFT(h z,j )⊙β z,j )

[0024]

[0025] Where FFT(·) and IFFT(·) represent Fourier transform and inverse Fourier transform, respectively, β z,j Represents the Fourier mask used to implement the frequency-dependent scaling factor sz , r represents the radius, r th Indicates the threshold frequency.

[0026] Furthermore, the road attribute information includes road type and road scale information.

[0027] Furthermore, the specific process of training the road structure generation model using the road structure dataset in step S3 is as follows:

[0028] First, a wide and deep network is used to encode and embed road attribute information. Then, the noise in the reverse process is estimated based on the conditional embedding c and the diffusion time step t. The purpose of training the diffusion model is to minimize the noise ∈ predicted by the model. θ (x t ,t,c) and Gaussian noise∈:

[0029]

[0030] In order to improve the smoothness of the generated road scene, a regularization term is introduced in the training objective. This regularization term quantifies the smoothness of the scene by calculating the distance between adjacent sampling points, so that the distance between adjacent sampling points in the real world road scene should be consistent with the distance between adjacent sampling points in the generated road scene. The smoothness loss term is used to adjust the difference of the generated samples:

[0031]

[0032] Among them, ∈ p+1 and ∈ p Respectively represent the p+1th point and the pth point in the added Gaussian noise ∈, and Represent the prediction noise ∈ θ The p+1th point and the pth point in ;

[0033] By combining the noise prediction loss and the smoothness loss, we get the hybrid objective loss function L(θ):

[0034] L(θ)=L mse (θ)+ωL s (θ)

[0035] Among them, ω is a hyperparameter used to balance the smoothness loss term L s and mean square error loss L mse To ensure that L s Will not dominate L mse .

[0036] Furthermore, the specific process of step S4 includes:

[0037] First, random Gaussian noise data is used as input, and the Road-UNet network is used to predict the noise that needs to be removed at each time step. The noise data predicted by the Road-UNet network is subtracted from the Gaussian noise data to obtain the data of the next time step. In the process of noise removal, the input road attribute information is used to guide the back diffusion denoising process. After T time steps of denoising process, the road scene data is finally iteratively generated.

[0038] Further, the reverse diffusion is defined as:

[0039]

[0040] in, and are the mean and variance of the back diffusion process at each time step t, And θ represents the parameters of the conditional diffusion model, and the corresponding calculation formulas for the mean and variance of the reverse diffusion process are:

[0041]

[0042] in, represents the predicted noise under condition t and road attribute c.

[0043] Furthermore, the road scene data includes control point coordinates and height data of each road, as well as lane number information.

[0044] Furthermore, the step S5 specifically uses a road evaluation function to evaluate the road scene data, and the road evaluation function is:

[0045]

[0046] in, It is the road continuity index, which is obtained by calculating the difference between the average curvature change rate of the real road scene and the generated road scene; is a road rationality index used to evaluate whether there are overlapping roads in the generated road scene. It is quantified by the ratio of overlapping roads to the total number of roads. λ is an index used to balance the road continuity. and road rationality index The weighting coefficient of .

[0047] Compared with the prior art, the present invention has the following advantages:

[0048] The present invention constructs a road structure data set by collecting real road structure data; and establishes a road structure generation model based on the Unet network architecture and wide-deep network architecture of FreeU; then trains the road structure generation model using the road structure data set to obtain a road structure diffusion generation model; and then inputs road attributes and random noise into the road structure diffusion generation model to output road scene data. The present invention utilizes the generation capability of the diffusion model to synthesize a variety of road layouts from white noise through a reverse denoising process, retaining the spatial characteristics of the real world, thereby being able to generate large-scale, realistic and diverse road scenes.

[0049] In the road structure generation model established by the present invention, a Unet network architecture based on FreeU is adopted, which integrates the road attention mechanism and the Road-UNet module. It can iteratively improve the noise estimation according to the real-world road data and the corresponding road attribute information to synthesize the final road scene; and introduce a smoothness regularization term to enhance the overall smoothness of the generated road; in addition, the balance between the backbone connection and the jump connection of the UNet network is designed to optimize. By adjusting the scaling factor of the backbone feature and the scaling factor of the jump feature, the details and quality of the generated road scene can be further improved, thereby generating a high-fidelity scene.

[0050] In the road structure generation model established by the present invention, a wide and deep network architecture is used to encode and embed road attribute information, including road type and road scale information. The integration of road structure data encoding and road attribute embedding can provide comprehensive information for the road structure generation model, ensuring the controllable generation of road scenes required for autonomous driving vehicle testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0052] Figure 2 is a schematic diagram of an application framework of an embodiment;

[0053] Figure 3 Schematic diagram of a visual diffusion generation process of a road scene in an embodiment;

[0054] Figure 4 A schematic diagram showing a comparison between a real road scene and a generated road scene in an embodiment;

[0055] Figures 5a to 5d A schematic diagram of a road scene generated in the CARLA simulation platform in an embodiment;

[0056] Figure 6a~6d The intelligent driving simulation test results for left-hand driving traffic on the VTD simulation platform in the embodiment;

[0057] Figures 7a to 7d It is a visualization result of the intelligent driving simulation test on right-hand-drive traffic on the CARLA simulation platform in the embodiment. DETAILED DESCRIPTION

[0058] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0059] Example

[0060] like Figure 1 As shown, a method for generating a large-scale road simulation test scenario library comprises the following steps:

[0061] S1. Collect real road structure data and construct a road structure data set;

[0062] S2. Based on FreeU's Unet network architecture and wide-deep network architecture, a road structure generation model is established;

[0063] S3, using the road structure data set to train the road structure generation model to obtain a road structure diffusion generation model;

[0064] S4, inputting road attributes and random noise into a road structure diffusion generation model, and outputting road scene data;

[0065] S5. Evaluate, sort and filter the output road scene data to build a road scene library for autonomous driving vehicle simulation testing.

[0066] This embodiment applies the above solution to build Figure 2 The application framework shown in the figure mainly includes a road structure generation model based on an enhanced conditional diffusion model, and a scene-level evaluation and screening module. The integration of road structure data encoding and road attribute embedding provides comprehensive information for the road scene generation model, ensuring the controllable generation of road scenes required for autonomous driving vehicle testing. The road attention mechanism and Road-UNet module are integrated into the road structure generation model, which can iteratively improve noise estimation based on real-world road data and corresponding attribute information to synthesize the final road scene. In addition, a smoothness regularization term is introduced to enhance the overall smoothness of the generated road.

[0067] In order to solve the problem that the generative model may generate unrealistic scenarios, this solution introduces a scenario evaluation model to ensure the availability and authenticity of the simulation test scenario. The evaluated road scene is converted into the OpenDRIVE format for autonomous driving vehicle testing. This format has stronger compatibility and can be used in various autonomous driving simulation test software.

[0068] Specific content includes:

[0069] 1. Road scene diffusion generation

[0070] The goal of this scheme is to generate realistic and diverse road scenarios for autonomous vehicle testing. In this scheme, a road scenario is defined as a complex set of n interconnected roads, represented by x = {r 1 ,r 2 ,…,r n} to indicate that each road is a sequence of sampled road shape points. The jth road shape point on the i-th road is represented by a tuple To indicate that and Represents latitude and longitude respectively.

[0071] Diffusion probability models are a class of powerful generative models based on likelihood probability, which have received extensive attention and application in data generation tasks. Diffusion probability models use a gradual process, that is, noise is gradually added to the data through a forward diffusion process, and then the noise added at each step is estimated through a backward diffusion process, and the data is gradually attenuated by removing the noise to recover the noise-free data. In essence, the diffusion probability model embodies a Markov structure.

[0072] In the forward diffusion process, Gaussian noise N(·) of T time steps is gradually added to the original data x0~q(x0), so as to obtain latent variables x1,x2,…,x T , where T is an adjustable parameter, indicating the maximum diffusion step size. The forward diffusion process can be expressed as:

[0073]

[0074] in, represents the noise level, ε represents the noise sampled from the Gaussian distribution N(0,I), and β i ∈(0,1) is the variance timetable.

[0075] In the back diffusion process, the conditional diffusion model aims to obtain the Gaussian noise data. Reconstruct the original road scene data This scheme designs attributes such as road type and road size as conditional information to guide the diffusion model to generate road scenes that are noise-free and meet specific control requirements. The definition of the reverse diffusion process is as follows:

[0076]

[0077] in, and are the mean and variance of the back diffusion process at each time step t, And θ represents the parameters of the conditional diffusion model. The mean and variance of the reverse diffusion process are calculated as follows:

[0078]

[0079] in, represents the predicted noise under condition t and road attribute c.

[0080] 2. Road-UNet Network Architecture

[0081] To ensure accurate prediction of noise at each diffusion time step, the generative model needs to capture the spatial dependencies between road structures. Therefore, this scheme uses the Road-UNet architecture to build a network to predict the noise at each diffusion time step in the road diffusion model. The Road-UNet network consists of two main parts: down-sampling layers and up-sampling layers. Each layer contains multiple stacked residual network modules (Resnet modules) based on 1D-CNN. A transition module based on the attention mechanism is integrated between these Resnet modules. In order to enhance the noise learning at each diffusion time step, the Road-UNet network uses the sinusoidal embedding method to embed the time step information, which is then input into each Resnet module. In addition, the Road-UNet network also contains two shared parameter fully connected layers, which are added to the input of the Resnet module.

[0082] This scheme introduces multi-head FreeU operation to balance the mapping of backbone features and jump features in the neural network. The design of Road-UNet contains two scalar factors, namely, z The backbone feature scaling factor b z and for h z The jump feature scaling factor s z , where x z represents the backbone features in the zth block in the Road-UNet decoder, h z represents the skip connection feature in the zth block. For the backbone feature x z , according to the average feature map Adaptively adjust the scaling factor α z :

[0083]

[0084] Among them, b z is a scalar constant, Represents the feature map x z The jth channel in, B represents x z In order to retain some high-frequency details and enhance the realism of road scene generation, this design limits the scaling operation to x.z On half of the channel, as shown in formula (7).

[0085] For the frequency hopping characteristic h z , using spectrum modulation in the Fourier domain to selectively attenuate the low-frequency components of the frequency hopping feature:

[0086] h′ z,j =IFFT(FFT(h z,j )⊙β z,j ), (8)

[0087]

[0088] Where FFT(·) and IFFT(·) represent Fourier transform and inverse Fourier transform respectively. z,j Represents the Fourier mask used to implement the frequency-dependent scaling factor s z . r represents the radius, r th Indicates the threshold frequency.

[0089] 3. Controllable Road Structure Generation

[0090] In order to generate controllable, realistic and diverse road scenes, this paper proposes a novel consistent conditional diffusion method to generate a large number of road scenes for autonomous vehicle testing. This paper uses a parameterized model To approximate the distribution of real-world road scenes, denoted as q(x0|c). This goal is achieved through a rigorous training and generation process as shown below.

[0091] Training process: During the training process, the wide-deep network is first used to encode and embed road attribute information, such as road type and road size. Subsequently, the noise in the reverse process is estimated based on the conditional embedding c and the diffusion time step t. The purpose of training the diffusion model is to minimize the noise ∈ θ (x t ,t,c) and Gaussian noise∈:

[0092]

[0093] In order to improve the smoothness of the generated road scenes, this scheme introduces a regularization term in the training objective. This term quantifies the smoothness of the scene by calculating the distance between adjacent sample points. Specifically, the distance between adjacent sample points in the real-world road scene should be consistent with the distance between adjacent sample points in the generated road scene. The smoothness loss term adjusts the differences of the generated samples to ensure that they closely match the ground truth values ​​observed in the real-world data. The smoothness loss term is expressed as:

[0094]

[0095] Among them, ∈ p+1 and ∈ p Respectively represent the p+1th point and the pth point in the added Gaussian noise ∈, and Represent the prediction noise ∈ θ The p+1th point and the pth point in .

[0096] By combining the noise prediction loss and the smoothness loss, a new hybrid objective loss function L(θ) is defined:

[0097] L(θ)=L mse (θ)+ωL s (θ), (12)

[0098] Among them, ω is a hyperparameter used to balance the smoothness loss term L s and mean square error loss L mse To ensure that L s Will not dominate L mse By adding a smoothness regularization term to the overall loss function, more realistic and smoother road scenes are generated.

[0099] Generation process: During the generation process, the road structure diffusion generation model will gradually transform from Gaussian noise to Generate realistic and diverse road scenes. Required road scenes It is iteratively generated from noise according to formulas (3) to (5). By integrating the Road-UNet network architecture and adjusting the scaling factor b of the backbone feature z and the scaling factor h of the jump feature z , further improving the details and quality of the generated road scenes.

[0100] 4. 3D Road Scene Generation

[0101] This embodiment further designs a three-dimensional road structure generation method according to the "Road Structure Design Specification". The maximum longitudinal slope varies with the design speed. For example, at a speed of 80 kilometers per hour, the maximum allowable slope is 5%. The centrifugal force F that a vehicle experiences when going up or down a ramp is c The maximum permissible limit should not be exceeded In order to ensure the safety and comfort of the vehicle when driving on a slope, the present invention formulates the following slope calculation formula in combination with the road curvature:

[0102]

[0103] ρ=min(ρ c ,ρ max ), (14)

[0104] Among them, ρ is the slope, m is the vehicle mass, v is the design speed limit of the road, and r is the road curvature radius.

[0105] 5. Scenario-level road assessment and screening module

[0106] Since the generative model may generate road scenes that do not meet specific standards, this paper proposes a road scene evaluation and screening model to solve this problem. The model includes two key indicators: road continuity index and road rationality index.

[0107] Specifically, the road continuity index is obtained by calculating the difference between the average curvature change rate of the real road scene and the generated road scene, which is expressed as The road rationality index evaluates whether there are overlapping roads in the generated road scene, which is quantified by the ratio of overlapping roads to the total number of roads, expressed as The final road evaluation function can be expressed as:

[0108]

[0109] Here, λ is a weighting factor used to balance the two terms.

[0110] The road scene generation method proposed in this solution can be used to generate all types of road scenes in the world for use in autonomous driving vehicle testing. In order to verify the effectiveness of this solution, this embodiment uses three real-world road data sets to conduct a large number of experiments. Considering that the intersection is one of the most complex road scenes, in order to more effectively demonstrate the proposed method, this embodiment uses the intersection road scene as an example for experimental verification and analysis.

[0111] Experimental setup:

[0112] The road structure data of major intersections in Singapore, the United Kingdom, and Japan were automatically collected from OpenStreetMap using the OSMnx tool. The statistical summary of the dataset is provided in Table 1. For each road structure dataset, data of twelve roads near the center of each intersection were extracted and resampled to a fixed length using interpolation.

[0113] Table 1 Statistics of the collected real-world road dataset

[0114] Dataset latitude longitude Number of road scenes Average road length Singapore (1.179，1.478) (103.599，104.054) 26 402 1075.227m U.K. (50.000，58.000) (-8.000，2.000) 71 484 901.561m Japan (34.000，45.551) (129.000，153.987) 149 561 1036.370m

[0115] The main purpose of generating road scenes is to test self-driving cars, so it is necessary to create realistic and diverse road scenes. To comprehensively evaluate the quality of the generated simulated road scenes, the key is to evaluate the similarity and smoothness between the simulated road scenes and the real-world road scenes. In terms of similarity indicators, Hausdorff distance and Jensen-Shannon divergence are used to measure the quality of the generated road scenes. For the smoothness indicator, the square integral of the second-order derivative is used as a measure of smoothness.

[0116] Table 2 lists the performance comparison of the road structure diffusion generation model trained in this scheme and other existing comparison methods on three real-world datasets.

[0117] Table 2 Statistics of road structure generation performance of different methods in road datasets from three different countries

[0118]

[0119] This embodiment uses each generation method to generate 5000 road scenes and calculates all relevant indicators. The experimental results effectively demonstrate the advantages of this solution. And the conditional version of the road structure diffusion generation model is better than the unconditional version because it incorporates road attribute information into the road scene generation process. This integration enables the road structure diffusion generation model to more effectively consider factors such as road structure type and spatial distribution, which are crucial for accurately generating road structures. The Hausdorff distance indicator in the road authenticity metric also confirms this superiority, in which the conditional version of the road structure diffusion generation model is significantly better than other models. In addition, even if the Road-UNet network structure, smoothness loss or scene evaluation module is abandoned, the road structure diffusion generation model can produce satisfactory results. Nevertheless, the complete road structure diffusion generation model is superior to its variant without Road-UNet in terms of both Hausdorff distance indicator and smoothness indicator. The Road-UNet network structure in the road structure diffusion generation model is designed for spatial structure modeling and multi-scale feature fusion, and can accurately predict the noise level during the denoising process, thereby generating high-quality road scenes. The road structure diffusion generation model is also superior to the version without smoothness loss in terms of Hausdorff distance and smoothness indicators. This is because the smoothness loss in the road structure diffusion generation model is specifically used to enhance the smoothness of the road structure, making the generated road smoother. In addition, the scene evaluation module can effectively filter out unreasonable road scenes, thereby improving the authenticity and consistency of the final road scene library.

[0120] In addition, this embodiment also visualizes the generated results to better demonstrate the performance of the road structure diffusion generation model. Figure 3Qualitative results of the generative process of the Road Structure Diffusion Generative Model are presented. The Road Structure Diffusion Generative Model can generate road scenes based on the specified desired scene type (e.g., "three three-way intersections") input by the user. The model first samples noise from a Gaussian distribution with k=500. Through a progressive denoising process, the Road Structure Diffusion Generative Model refines the sampled noise until it converges to k=0. Comparative analysis with real-world road scenes shows that the Road Structure Diffusion Generative Model can effectively capture the complex characteristics of real-world road scenes and generate realistic and diverse road configurations.

[0121] Figure 4 Real-world road scenes and generated three-dimensional road scenes are shown. The road structure diffusion generation model can generate specific types of road scenes based on road attributes and condition information to meet the requirements of autonomous vehicle testing. The first row of the figure shows real-world road scenes, and the following rows show various road test scenes generated by the road structure diffusion generation model. The first column shows intersection scenes generated using specific condition information, including a four-way intersection and two three-way intersections. The second column shows a taxi pick-up and drop-off scene. The third column shows the generated roundabout scene. Finally, the fourth column shows the generated overpass scene.

[0122] from Figure 4 It can be seen that the road structure diffusion generation model can effectively learn the distribution characteristics of real-world road structures and generate three-dimensional scenes that are very similar to these structures, that is, it can generate specific types of road scenes based on road attributes and condition information, thereby meeting the requirements of autonomous vehicle testing. In addition, the road structure diffusion generation model also greatly enhances the diversity of simulation test road scenes, so that the performance of autonomous vehicles on various road types can be comprehensively evaluated. The integration of controllable condition information enables the road structure diffusion generation model to accurately generate specified types of three-dimensional road scenes. This powerful function highlights the versatility of the road structure diffusion generation model in generating meaningful road scenes based on specific conditions.

[0123] In order to further verify the effectiveness of the three-dimensional road scene generated by DiffRoad in the test of autonomous driving vehicles, this embodiment uses the generated road scene to conduct more autonomous driving vehicle test experiments based on digital twins. The intelligent driving test experiments are carried out using the CARLA simulation platform and the VTD simulation platform respectively to prove the universality of the scene generated by the road structure diffusion generation model of this scheme. First, the OpenDRIVE scene generated by the road structure diffusion generation model is imported into CARLA and VTD as the test environment for autonomous driving vehicles. In order to prove the universality and effectiveness of the road scene generated by the road structure diffusion generation model, a general autonomous driving model is used for evaluation on the CARLA and VTD platforms.

[0124] Figures 5a to 5d The following is an overview of the road scene generated by the road structure diffusion generation model in the CARLA simulation platform. Specifically, Figure 5a The intersection scene is shown. Figure 5b It shows the scene of taxi passengers getting on and off. Figure 5c The scene of the roundabout is shown. Figure 5d Overpass scenes are shown. These 3D scenes effectively demonstrate the ability of the road structure diffusion generation model to create a variety of 3D road structures suitable for autonomous vehicle testing. Figure 6a~6d The results of intelligent driving simulation tests on left-hand drive traffic on the VTD simulation platform are presented. Figure 6a In the video, the VTD autonomous driving vehicle passed through different intersections and clashed with buses and cars at different intersections. Figure 6c The video shows an autonomous vehicle operating in a taxi pickup and drop-off scenario, where it had a trajectory conflict with a truck when leaving the taxi pickup and drop-off area. Figure 6c The video shows an autonomous vehicle navigating a roundabout scenario, where it collides with surrounding vehicles when entering and exiting the roundabout. Figure 6d A scene showing a self-driving car passing through an overpass. Figures 7a to 7d The visualization results of the intelligent driving simulation test on right-hand-drive traffic on the CARLA simulation platform are shown. Figure 7a In the video, the CARLA autonomous vehicle passes through multiple intersections. Figure 7b The CARLA autonomous vehicle was shown in a taxi pick-up and drop-off scenario. The simulation test showed that the autonomous vehicle collided with the edge of the road at 11.8 seconds. This also proves that the scenarios generated by the road structure diffusion generation model can effectively test the safety of autonomous vehicles in various road shapes. Figure 7c The test results of the autonomous vehicle in the roundabout scene were shown. Figure 7d It shows a scene of a self-driving car passing through an overpass.

[0125] The above experimental results show that the road scenes generated by the road structure diffusion generation model can effectively evaluate the safety of autonomous vehicles under different road conditions. In addition, the road scenes generated by the road structure diffusion generation model can also be effectively used for autonomous vehicle testing, overcoming the limitations of existing methods in the number and types of road scenes. Unlike existing studies that can only generate road images, the road structure data generated by the road structure diffusion generation model can be automatically converted into three-dimensional road scene data in the OpenDRIVE format for intelligent driving testing. In addition, the proposed method is also highly versatile and scalable, can generate any type of road scene, and is compatible with various simulation test platforms. The generated comprehensive scene library greatly speeds up the testing of autonomous vehicles and lays the foundation for future performance improvements in autonomous driving technology.

[0126] In summary, this scheme proposes a method for generating a large-scale static traffic scene library based on a diffusion model, aiming to generate controllable high-fidelity three-dimensional road scenes. This scheme uses the generation ability of the diffusion model to synthesize a variety of road layouts from white noise through an inverse denoising process, retaining the spatial characteristics of the real world. In order to improve the quality of the generated scenes, the Road-UNet architecture is designed, and the balance between the backbone connections and jump connections of the UNet network is optimized to generate high-fidelity scenes. Experimental results on multiple real-world datasets show that this scheme is able to generate realistic and smooth road structures while maintaining the original distribution. In addition, the generated scenes can be fully automatically converted to the OpenDRIVE format, which facilitates general autonomous driving vehicle simulation testing. This scheme can provide a rich and diverse scene library for large-scale autonomous driving vehicle testing.

Claims

1. A method for generating a large-scale road simulation test scenario library, characterized in that: The following steps are involved: S1. Collect real road structure data and construct a road structure data set; S2. Establish a road structure generation model based on FreeU's Unet network architecture and wide-deep network architecture; S3, using the road structure data set to train the road structure generation model to obtain a road structure diffusion generation model; S4, inputting road attributes and random noise into a road structure diffusion generation model, and outputting road scene data; S5. Evaluate, sort and filter the output road scene data to build a road scene library for autonomous driving vehicle simulation testing.

2. The method for generating a large-scale road simulation test scenario library according to claim 1, characterized in that: The step S1 includes the following process: S11. Collecting real road structure data by web crawling; S12. For the collected real road structure data, the coordinate data of the corresponding road shape control points are extracted, and the interpolation method is used to resample the data to a fixed length to construct a road structure data set.

3. The method for generating a large-scale road simulation test scenario library according to claim 1, characterized in that: In the road structure generation model established in step S2, the Unet network architecture is used to predict the noise of each diffusion time step, and the wide-deep network architecture is used to encode and embed road attribute information, and the road attribute information includes road type and road scale information.

4. The method for generating a large-scale road simulation test scenario library according to claim 3, characterized in that: The Unet network architecture includes multiple Road-UNet networks connected in sequence, each Road-UNet network includes a down-sampling layer and an up-sampling layer, the down-sampling layer and the up-sampling layer each include multiple stacked Resnet modules based on 1D-CNN, a transition module based on an attention mechanism is integrated between the multiple Resnet modules, and the input end of the Resnet module is connected to two fully connected layers with shared parameters.

5. The method for generating a large-scale road simulation test scenario library according to claim 4, characterized in that: The Unet network architecture introduces multi-head FreeU operations to balance the mapping of backbone features and jump features in the neural network, where Road-UNet contains two scalar factors: z The backbone feature scaling factor b z and for h z The jump feature scaling factor s z , where x z represents the backbone features in the zth block in the Road-UNet decoder, h z represents the skip connection feature in the zth block in the Road-UNet decoder. For the backbone feature x z , according to the average feature map Adaptively adjust the scaling factor α z : Among them, b z is a scalar constant, Represents the feature map x z The jth channel in, B represents x z The total number of channels in , ⊙ represents the element-wise product; For the frequency hopping characteristic h z , using spectrum modulation in the Fourier domain to selectively attenuate the low-frequency components of the frequency hopping feature: h′ z,j =IFFT(FFT(h z,j )⊙β z,j ) Where FFT(·) and IFFT(·) represent Fourier transform and inverse Fourier transform respectively, β z,j Represents the Fourier mask used to implement the frequency-dependent scaling factor s z , r represents the radius, r th Indicates the threshold frequency.

6. The method for generating a large-scale road simulation test scenario library according to claim 3, characterized in that: The specific process of training the road structure generation model using the road structure data set in step S3 is: First, a wide and deep network is used to encode and embed road attribute information. Then, the noise in the reverse process is estimated based on the conditional embedding c and the diffusion time step t. The purpose of training the diffusion model is to minimize the noise ∈ predicted by the model. θ (x t ,t,c) and Gaussian noise∈: In order to improve the smoothness of the generated road scene, a regularization term is introduced in the training objective. This regularization term quantifies the smoothness of the scene by calculating the distance between adjacent sampling points, so that the distance between adjacent sampling points in the real world road scene should be consistent with the distance between adjacent sampling points in the generated road scene. The smoothness loss term is used to adjust the difference of the generated samples: Among them, ∈ p+1 and ∈ p Respectively represent the p+1th point and the pth point in the added Gaussian noise ∈, and Represent the prediction noise ∈ θ The p+1th point and the pth point in ; By combining the noise prediction loss and the smoothness loss, we get the hybrid objective loss function L(θ): L(θ)=L mse (θ)+ωL s (i) Among them, ω is a hyperparameter used to balance the smoothness loss term L s and mean square error loss L mse To ensure that L s Will not dominate L mse .

7. The method for generating a large-scale road simulation test scenario library according to claim 6, characterized in that: The specific process of step S4 includes: First, random Gaussian noise data is used as input, and the Road-UNet network is used to predict the noise that needs to be removed at each time step. The noise data predicted by the Road-UNet network is subtracted from the Gaussian noise data to obtain the data of the next time step. In the process of noise removal, the input road attribute information is used to guide the back diffusion denoising process. After T time steps of denoising process, the road scene data is finally iteratively generated.

8. The method for generating a large-scale road simulation test scenario library according to claim 7, characterized in that: The back diffusion is defined as: in, and are the mean and variance of the back diffusion process at each time step t, And θ represents the parameters of the conditional diffusion model, and the corresponding calculation formulas for the mean and variance of the reverse diffusion process are: in, represents the predicted noise under condition t and road attribute c.

9. The method for generating a large-scale road simulation test scenario library according to claim 7, characterized in that: The road scene data includes the control point coordinates and height data of each road, and the number of lanes information.

10. The method for generating a large-scale road simulation test scenario library according to claim 1, characterized in that: The step S5 specifically uses a road evaluation function to evaluate the road scene data, and the road evaluation function is: in, It is the road continuity index, which is obtained by calculating the difference between the average curvature change rate of the real road scene and the generated road scene; is a road rationality index used to evaluate whether there are overlapping roads in the generated road scene. It is quantified by the ratio of overlapping roads to the total number of roads. λ is an index used to balance the road continuity. and road rationality index The weighting coefficient of .

Images