A controllable generation method and system of topological road network for autonomous driving simulation

Abstract

The present invention discloses a controllable generation method and system of a topological road network for autonomous driving simulation. First, the present invention encodes spatial position constraints and inputs them into a generator to generate a road network image under the constraints. Then, a discriminator judges the authenticity of the generated road network image and whether it complies with the constraints of a spatial position feature map. At the same time, a differentiable topological structure extractor is used to extract the topological structure of the generated road network image for a topological attribute reward network to calculate the topological attribute score of the generated road network, and the topological attribute score is used to guide the generator to generate a topological road network that meets specific topological attributes. Based on a generative artificial intelligence model, the present invention simultaneously models and learns the road network spatial position information and topological features in the training data, and generates a large number of topological road network models with strong connectivity and meeting expected attributes under given spatial position constraints.

Description

Technical Field

[0001] The present invention belongs to the fields of computer technology and intelligent transportation technology, and specifically relates to a controllable generation method and system of a topological road network for autonomous driving simulation. Background Art

[0002] With the development of intelligent connected transportation and autonomous driving technologies, the construction of smart cities needs to consider more the needs of dedicated road networks for autonomous driving. Generating a friendly and efficient road network layout for autonomous driving travel, simulating and analyzing the operation of the road network in a simulation environment, and iteratively optimizing the generated road network are of great significance to improving traffic efficiency and reducing traffic congestion. In addition, the road network generation technology automatically generates a large number of road network models at a very low cost, providing a large amount of training and test data for artificial intelligence-based autonomous driving algorithms, which is of great significance for the simulation test of autonomous driving vehicles.

[0003] However, manually designing a road network with specific features is a time-consuming and tedious operation. In recent years, researchers have begun to design automated methods to generate a large number of road networks. Traditional programmatic road network generation methods usually semi-automatically generate road networks based on a set of rules and parameters. Such methods require a lot of manual operations and parameter settings, and the generated road network models are not diverse enough, and it is impossible to generate a certain number of road networks for selection in subsequent tasks. In recent years, generative artificial intelligence has received widespread attention, forming deep generative models such as variational autoencoders, generative adversarial networks, and diffusion models. Road network generation methods based on deep generative models can be summarized as graph-based methods and image-based methods. Existing graph-based generation methods can better maintain the connectivity of road networks, but cannot encode the spatial characteristics of road networks, resulting in the inability to generate reasonable road networks under given location constraints (such as population density at different locations). Existing image-based generation methods can better model spatial location information, but ignore the topological characteristics of road networks, resulting in fragmented and poorly connected generated road networks. In addition, these methods are unable to generate road networks with expected properties (such as high traffic convenience), making it difficult to provide suitable solutions for road network planning needs in areas such as intelligent transportation and autonomous driving. Summary of the invention

[0004] In order to solve the shortcomings of the prior art and realize the controllable generation of a road network with strong connectivity, satisfying specified spatial constraints and having expected topological properties, so that the road network can be better applied to the purpose of autonomous driving simulation testing, the present invention adopts the following technical solutions:

[0005] A controllable generation method of a topological road network for autonomous driving simulation comprises the following steps:

[0006] Step S1: Encode the spatial position constraint condition to obtain an encoded spatial position feature map;

[0007] Step S2: The generator uses the spatial position feature map as a constraint condition to generate a road network image that satisfies the spatial position constraint;

[0008] Step S3: The discriminator determines the authenticity of the generated road network image and whether it conforms to the constraints of the spatial position feature map;

[0009] Step S4: inputting the generated road network image into a differentiable topology structure extractor at the same time to extract the topology structure of the road network image end-to-end;

[0010] Step S5: Input the road network topology structure into the topology attribute reward network, calculate the topology attribute score of the road network topology structure, and guide the generator to generate a road network that meets specific topology attributes.

[0011] Furthermore, in step S1, an autoencoder A is constructed to obtain the spatial position constraint graph S and encode it into a low-dimensional feature graph F. S ; Then the feature map F S It is also decoded and restored to the spatial position constraint graph S′, and the autoencoder A is trained based on the similarity between the spatial position constraint graph S and the spatial position constraint graph S′.

[0012] Furthermore, in step S2, a road network generator G is constructed, and the Transformer module is used as the backbone network to model the global consistency of the road network image and obtain the spatial position feature map F S , converted into a multi-channel marker block (h*w marker block tokens with C channels) and a learnable position encoding marker block by a multi-layer perceptron, and the marker block is sent to several Transformer modules and upsampling modules to gradually generate a road network image I of size H*W*1 that satisfies the spatial position constraint S; among them, the Transformer module is used to model the local detail features and global consistency of the image, and the upsampling module is used to expand the resolution of the image.

[0013] Furthermore, in order to balance local details and global computational efficiency, in the process of gradually generating images, for images with a size exceeding a threshold The intermediate image is divided into several grids, and then each grid block is regarded as a labeled block instead of each pixel as a labeled block, so as to reduce the number of labeled blocks, reduce the calculation amount of the network model, and improve the calculation efficiency;

[0014] The upsampling module uses a bicubic interpolation method to expand the image resolution of the low-resolution intermediate image and keeps the number of channels C of the image unchanged; for the high-resolution intermediate image, a pixel shuffle method is used to expand the image resolution and reduce the number of channels of the image to reduce the amount of calculation in the model learning process; finally, after the road network image size generated by multiple Transformer modules and upsampling modules is H*W*(16 / C), a linear projection layer is used to convert it into a road network image I with a corresponding size of H*W*1.

[0015] Furthermore, in step S3, a road network discriminator D is constructed, including a group of Transformer modules and a detection head. The generated road network image I is divided into multiple image marker blocks of different scales through a linear embedding layer and sent to the corresponding Transformer modules of multiple stages respectively. The detection head includes a multi-layer perceptron, which makes a binary classification of whether the generated image is true or false. If the generated road network image is consistent with the distribution of the real road network data and conforms to the spatial position feature map F S If the constraint is met, the judgment is true, otherwise it is false.

[0016] Furthermore, the road network image I is divided into three image marker blocks of different scales and sent to three stages of Transformer modules respectively. The Transformer module of the first stage takes the size as the first scale. The image marker block is taken as input, and after processing, it is sent to the downsampling module, and the output size is the second scale Then, the label block is connected with the label block of the input image with the same scale and sent to the Transformer module of the second stage; similarly, the size of the third scale is obtained The image marker block is sent to the third stage; after the third stage, the scale of the output marker block is kept unchanged, and a classification marker block is attached to assist the detection head in carrying out the classification task.

[0017] In general, the road network topology conforms to the expression specification of the graph data structure, and is composed of a set of nodes and a set of connection relationships (edge ​​sets). Although some traditional image morphology methods can extract topological expressions in binary images, these methods are not differentiable and cannot be integrated into the intermediate module of the neural network model. Therefore, the method of the present invention designs a differentiable image topology extractor to extract and generate the topological structure in the road network image in an end-to-end manner, and can back-propagate errors during the training process of the neural network model.

[0018] Further, in step S4, the differentiable topological structure extractor E includes an encoding module, a decoding module, a node coordinate detection head and a connection relationship detection head. The decoding module takes the encoding feature map output by the encoding module, a plurality of node coordinate marking blocks, and a connection relationship marking block as input. The decoded node coordinate marking blocks and the connection relationship marking blocks are simultaneously sent to the node coordinate detection head and the connection relationship detection head. The node coordinate detection head is composed of a plurality of groups of multi-layer perceptrons to predict the coordinates between these nodes. The connection relationship detection head is also composed of a plurality of groups of multi-layer perceptrons. Each group of multi-layer perceptrons takes any two decoded node coordinate marking blocks and one connection relationship marking block as input to predict the connection relationship between these two nodes.

[0019] The encoding module and decoding module are composed of a deformable attention module based on Transformer, so that the query operation in Transformer focuses on a small range of spatial features determined by a learnable offset near a reference point, which improves the model convergence speed and reduces the computational complexity of the query operation. The formula is as follows:

[0020]

[0021] Among them, x represents the input of the module, M represents the number of attention heads, k represents the sequence number of the sampled key, K represents the total number of sampled keys, and the number of samples should be much smaller than the image feature size. mqk and A mqk They represent the position offset of the mth attention head, the kth sampling point and the normalized attention weight, respectively. q represents the reference point, W m It is the linear transformation of the result after attention is applied to value (value) to obtain the output results of different heads, W′ m Used to transform x into a value.

[0022] Furthermore, in step S5, the extracted road network topology structure is input into the topology attribute reward network R. The reward network R first encodes the road network topology structure and converts it into a graph structure embedding vector. The embedding vector is then sent to a regression module composed of a multi-layer perceptron and an activation function to calculate the topological attribute (such as high traffic convenience) score of the road network topology structure. The attribute score is used as a supervision signal of the road network generator G. Based on the attribute score, the road network generator G is controlled to generate a road network that satisfies specific topological attributes.

[0023] Furthermore, the reward network R uses a spatially-aware enhanced graph convolutional neural network to encode the input road network topology structure, and uses the distance between nodes as a feature in the encoding process to capture the topological information of the road network graph structure and the spatial distance information between nodes. The encoding process formula is as follows:

[0024]

[0025] Among them, c i and c j They are node v i and v j The coordinate vector in the image coordinate system, f j Represents each node v j The feature vector of is composed of the normalized coordinates and average direction angle of its neighbor nodes. i All neighbors of v j , by aggregating the features of these nodes into To construct graph structure embedding vector where u ij represents the learnable weights;

[0026] The optimization goal of the entire road network generation process is composed of the WGAN-GP loss function related to the spatial position of the road network and the topological attribute reward score, so as to simultaneously model and learn the spatial position information and topological features of the road network in the training data, and then generate a large number of topological road network models with strong connectivity, satisfying the given spatial position constraints and satisfying the expected attributes. The formula is as follows:

[0027] L(x)=(1-λ)L w (x)+λL r (x) = (1-λ)L w (x)+λ(1-score(E(x)))

[0028] Where x represents the generated image, E(x) represents the corresponding road network topology output by the differentiable topology extractor, score(·) represents the scoring function, and L w represents the WGAN-GP loss function related to the spatial location of the road network, L r represents the topological attribute reward loss function, and the balance parameter λ is used to balance the strength between the two.

[0029] The loss function formula is as follows:

[0030]

[0031] Among them, P G and P data They represent the data distribution generated by the road network generator D and the distribution of real data, respectively. Represents sampling from these distributions, β represents the coefficient of the gradient penalty, and the constant k is a constraint on the discriminator gradient, requiring that the gradient norm during the learning process does not exceed the constant k to improve the stability of the training process and alleviate the mode collapse problem.

[0032] A controllable generation system of a topological road network for autonomous driving simulation, comprising a data acquisition and processing module, a spatial position condition encoding module, a topological road network generation module, a road network format conversion module, and a road network quantitative evaluation module;

[0033] The data collection and processing module collects and processes the road network data in the real world to form a data set for training the controllable generation of topological road networks. The execution process of collecting and processing the data is as follows:

[0034] (1) Collect road network data in the real world from online map websites, and then rasterize it into a black-and-white binary image. Specifically, the method steps for rasterizing road network data in the real world are as follows: first, select the geographic information coordinate system and determine the image scaling (i.e., the actual distance represented by one pixel in the image). Then, expand the road network segments in the real world into different road widths according to the road network level. In the corresponding rasterized image, the road network part is white and the background part is black.

[0035] (2) At the same time, the population density map, land use type map, and river water area map of the corresponding area of ​​the road network are collected to form a spatial location constraint map.

[0036] (3) The rasterized image is divided into several image blocks. During the division process, there must be a certain overlap ratio between the image blocks.

[0037] (4) For each image block, the black and white binary image is processed into a graph structure using morphological methods, and redundant nodes are removed and simplified into a road network topology structure. Specifically, the black and white binary image is first node mapped to distinguish the black background, white road network, and road network nodes. Then the adjacent nodes are grouped into a node domain, and each node domain is indexed and encoded. Secondly, starting from the node, the lines in the node neighborhood are traversed. If a node is found twice in the neighborhood, the line is used as the connection between the two nodes. Finally, the center point of the node domain is found, and all nodes and their connection relationships are output. For these nodes and connection relationships, this method simplifies all nodes with a degree of 2 and removes redundant nodes. Specifically, if the angle between the two edges associated with a node with a degree of 2 is greater than the set threshold, the redundant node is removed.

[0038] (5) Calculate the topological properties of the road network topology (such as traffic convenience, road network direction sequence, etc.).

[0039] (6) The road network image, the corresponding spatial position constraint graph, the corresponding topological structure, and the corresponding topological attributes are combined into a data set, and their rationality and richness are checked for training of the controllable generation model of the topological road network.

[0040] The spatial position condition encoding module encodes the collected spatial position constraint conditions to obtain an encoded spatial position feature map;

[0041] Specifically, during the model training process, the encoding module in the autoencoder is first used to encode the spatial position constraint map S into a feature map with a lower dimension, and then the decoding module is used to restore the encoded feature map to the spatial position constraint map S′ input to the encoder, and the highest similarity between S and S′ is used as the optimization goal. During the model inference process, only the encoding module in the autoencoder is used to process the feature map of the spatial position constraint condition;

[0042] The topological road network generation module uses the spatial position feature map as a constraint condition, and generates a road network image that satisfies the spatial position constraint through a generator; the generated road network image and the feature map of the spatial position constraint condition are simultaneously sent to the discriminator, and the road network image is discriminated for consistency with the real road network data and whether it meets the constraint of the spatial position feature map, and the authenticity of the road network image generated by the generator is judged, and whether it meets the constraint of the spatial position feature map; in addition, the generated road network image is simultaneously input into a differentiable topological structure extractor to extract the topological structure of the generated road network image end-to-end, so that the topological attribute reward network calculates the topological attribute score of the generated road network, and based on the topological structure, a road network that meets specific topological attributes is generated, that is, the topological attribute score is used to guide the generator to generate a topological road network that meets specific topological attributes;

[0043] The road network format conversion module divides the generated road network topology into sub-graphs, and numbers each road in the road network and adds virtual geographic information to convert it into a road network format that meets the requirements of the autonomous driving simulation environment. The execution process of converting the generated topological road network into the autonomous driving map format is as follows:

[0044] (1) Divide the road network topology into subgraphs so as to number each road in the road network. Specifically, this method stipulates that the subgraph must satisfy the following three properties: first, the subgraph is an acyclic graph; second, the degree of each node in the subgraph does not exceed 2; third, the angle formed by the two edges associated with any node in the subgraph is not less than a threshold. Then, through the breadth-first search method, the road network topology data is divided into several subgraphs and numbered to distinguish each road.

[0045] (2) Generate road features and road widths based on road numbers and road attributes (main roads, secondary roads, or roads within residential areas, etc.). This method refers to the OSM road specification (i.e., XML format specification) and generates four types of features, including nodes (Node) representing geographic locations, channels (Way) representing linear features or planar features, relationships (Relation) between existing nodes and channels, and tags (Tag) used to store metadata of map features. In the process of generating these features, except for nodes, the content of other features can be empty, but the feature field index must exist. Specifically, this method first stores the road topology structure after numbering and grouping as data in a CSV file. Then, this method defines a basic XML format template and adds XML characters representing the generated nodes and channels to it. Then, this method extracts the identification, longitude, and latitude information of each road node in turn, generates XML characters for each node, and generates XML text for the road based on the road grouping number information.

[0046] (3) Convert the generated XML format file into a file that complies with the specifications of the autonomous driving simulation environment and check its legitimacy. Specifically, this method specifies the origin of the coordinate system for the generated road network, and then converts the XML file into the corresponding format file based on the script provided by the autonomous driving simulation software CARLA, and checks the legitimacy of the conversion process. Similarly, in order to meet the requirements of the autonomous driving road network simulation software SUMO, this method converts it into a traffic network description file and a traffic flow simulation model file.

[0047] The road network quantitative evaluation module calculates traffic performance indicators (including road connectivity, traffic convenience, generation quality, and generation diversity) for the generated topological road network to verify the rationality of the generated topological road network model and the effectiveness of the topological road network generation method, that is, to verify that the topological road network generation method proposed in the present invention has the ability to generate a road network with strong connectivity, satisfying specified spatial constraints, and having expected topological attributes.

[0048] The advantages and beneficial effects of the present invention are:

[0049] (1) The present invention proposes a controllable generation method for a topological road network, which aims to controllably generate a road network with strong connectivity, satisfying specified spatial constraints, and having expected topological properties, and is applied to autonomous driving simulation tests. In view of the problem that existing road network generation technologies are difficult to maintain the topological connectivity of the road network while modeling spatial location information, and are difficult to generate a road network with expected properties, the present invention is based on a generative artificial intelligence model, and simultaneously models and learns the road network spatial location information and topological features in the training data, and generates a large number of topological road network models with strong connectivity and satisfying expected properties (such as high traffic convenience) under given spatial location constraints.

[0050] (2) The present invention designs a differentiable image topology structure extractor and a topology attribute reward network. The extractor extracts the topology structure in the generated road network image in an end-to-end manner, and can back-propagate errors during the training process of the entire topology road network generation network model to assist the generator in generating a topology road network model that meets the expected attributes. The topology attribute reward network calculates the topology attribute score of the road network topology structure, and the output topology attribute score is used as a supervisory signal of the generator to control the generator to generate a road network that meets specific topology attributes.

[0051] (3) The present invention provides a controllable topological road network generation system for autonomous driving simulation, wherein the data acquisition and processing module included therein can efficiently and comprehensively collect the training data required by the controllable topological road network generation method; the road network format conversion module included therein can accurately convert the generated topological road network into a road network format that meets the requirements of the autonomous driving simulation environment; the road network quantitative evaluation module included therein can evaluate the quality of the generated topological road network in a quantitative manner, thereby verifying the effectiveness of the controllable topological road network generation method. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 Schematic diagram of the deep neural network model structure of the method in the embodiment of the present invention.

[0053] Figure 2 It is a schematic diagram of the structure of the spatial position conditional coding module in an embodiment of the present invention.

[0054] Figure 3 Schematic diagram of the structure of a differentiable topological structure extractor of the method in an embodiment of the present invention.

[0055] Figure 4 It is a schematic diagram of the system module structure in an embodiment of the present invention.

[0056] Figure 5 It is the intermediate result and the corresponding topological network effect diagram generated in the embodiment of the present invention.

[0057] Figure 6 It is a rendering effect diagram of the autonomous driving simulation environment for generating a topological road network in an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the present invention, and is not used to limit the present invention.

[0059] like Figure 1As shown in the figure, a controllable generation method of topological road network for autonomous driving simulation is composed of five parts: autoencoder A, road network generator G, discriminator D, differentiable topological structure extractor E, and topological attribute reward network R, including the following steps:

[0060] Step S1: Based on the autoencoder A, encode with the given spatial position constraint as input to obtain a feature map of the encoded spatial position constraint.

[0061] The autoencoder A consists of two modules, encoding and decoding. The input is a spatial position constraint condition map S consisting of a population density map, a land use type map, and a river water map. The encoding module in the autoencoder A is used to encode these constraints and encode them into a spatial position constraint condition feature map F of size 8*8. S =A(S). Figure 2 As shown, the autoencoder A consists of two modules, encoding and decoding. During the parameter learning and training process of each module of the autoencoder, the optimization goal is to maximize the similarity between the spatial position constraint map S input to the encoder and the spatial position constraint map S′ after decoding, that is, the L1 distance d between S and S′ S =|SS′| is minimum.

[0062] Step S2: Using the spatial position feature map as a constraint condition, generate a road network image that satisfies the spatial position constraint.

[0063] The spatial position constraint feature map F of size 8*8 S As a constraint condition, the Transformer-based road network generator G is input to generate a road network image I that satisfies the spatial position constraint S. Specifically, the road network generator G is composed of a Transformer module as the backbone network, with the goal of modeling the global consistency of the road network image. The input spatial position feature map F SAfter being converted into 8*8 tokens with 64 channels and 1 learnable position encoding token block by a multi-layer perceptron. These token blocks are then fed into 6 Transformer modules and 5 upsampling modules to gradually generate a road network image of size 256*256*1. The purpose of the Transformer module is to model the local detail features and global consistency of the image, and the role of the upsampling module is to expand the resolution of the image. In order to balance local details and global computational efficiency, in the process of gradually generating images, for intermediate images with a size greater than 64*64, we divide them into several grids, and then regard each grid block as a token block (rather than each pixel as a token block) to reduce the number of token blocks, reduce the amount of computation of the network model, and improve computational efficiency. Correspondingly, for the upsampling module following the Transformer module, this method also adopts different upsampling methods. In the intermediate image with a resolution less than 64*64, this method uses the bicubic interpolation method to expand the image resolution and keep the number of channels of the image unchanged; in the intermediate image with a resolution greater than or equal to 64*64, this method uses the pixel shuffle method to expand the image resolution and reduce the number of channels of the image to reduce the amount of calculation in the model learning process. Finally, after the road network image size generated by multiple Transformer modules and upsampling modules is 256*256*4, this method uses a linear projection layer to convert it into a road network image I with a size of 256*256*1.

[0064] Step S3: road network identification is performed based on the consistency between the road network image and the real road network data, and whether the road network image meets the constraints of the spatial location feature map.

[0065] The spatial position feature map F SThe road network image I generated by the generator is input into the Transformer-based road network discriminator D, which performs the classification task. If the road network image generated by the generator G is consistent with the distribution of the real road network data and meets the constraints of the spatial position feature map, the discriminator makes a judgment that the generated road network image is true, otherwise it is judged to be false. Specifically, the discriminator D consists of 4 Transformer modules and 1 detection head. The generated road network image I is divided into three groups of image marker blocks of different scales after the linear embedding layer, and sent to the three-stage Transformer modules respectively. The Transformer module of the first stage takes the marker block of size (64×64)×96 as input, and after processing, it is sent to the downsampling module and outputs a marker block of size (32×32)×192. Then, the marker block is connected with the input image marker block of the same scale and sent to the Transformer module of the second stage. Similarly, the image marker block of size (16×16)×384 is sent to the third stage. After the third stage, the proposed method keeps the output label block of size (16×16)×384 unchanged and adds a classification label block of size 1×384 to assist the detection head in the classification task. The detection head is composed of a multi-layer perceptron and makes a binary classification of whether the generated image is true or false.

[0066] Step S4: The road network image is simultaneously input into the differentiable topology structure extractor E to extract the topology structure of the generated road network image end-to-end.

[0067] The road network image generated by the generator is simultaneously input into the differentiable topological structure extractor E to extract the topological structure of the generated road network image in an end-to-end manner. Specifically, the differentiable topological structure extractor E designed by the method of the present invention uses the encoding module, decoding module, node coordinate detection head and connection relationship detection head as core modules, takes the generated road network image I as input, and generates the node set corresponding to the road network image I and the connection relationship set between the nodes in an end-to-end manner. Among them, the encoding module and the decoding module are both composed of a deformable attention module based on Transformer, which can be specifically expressed as formula (1), where M represents the number of attention heads, k represents the serial number of the sampled keys, K represents the total number of sampled keys, and the sampling number should be much smaller than the image feature size, Δp mqk and A mqk They represent the position offset of the mth attention head, the kth sampling point and the normalized attention weight, respectively. q represents the reference point, W m It is a linear transformation of the result after attention is applied to value to obtain the output results of different heads, W′ m Used to transform x into a value.

[0068]

[0069] This module makes the query operation in Transformer focus on the small-scale spatial features determined by the learnable offset near the reference point, which improves the model convergence speed and reduces the computational complexity of the query operation. Figure 3 As shown, the decoder takes the encoded feature map output by the encoder, several node coordinate marker blocks, and one connection relationship marker block as input, and the decoded node coordinate marker blocks and connection relationship marker blocks will be sent to the node coordinate detection head and the connection relationship detection head at the same time. The node coordinate detection head is composed of 100 multi-layer perceptrons to support the prediction of coordinates between up to 100 nodes; each multi-layer perceptron in the connection relationship detection head takes any 2 decoded node coordinate marker blocks and 1 connection relationship marker block as input to predict the connection relationship between these 2 nodes. It is worth noting that the differentiable topological structure extractor E proposed in the present invention should be pre-trained, and the parameters of the differentiable topological structure extractor should remain fixed during the training of the entire road network generation model.

[0070] Step S5: Based on the topological structure, a road network having specific topological attributes is generated;

[0071] The extracted road network topology is input into the topological attribute reward network R. The reward network R first encodes the road network topology and converts it into a 64-dimensional graph structure embedding vector. The embedding vector is then sent to a regression module consisting of a multi-layer perceptron and an activation function to calculate the topological attribute score of the road network topology. The topological attribute score output by the reward network R will serve as a supervision signal for the generator G to control the generator G to generate a road network that satisfies specific topological attributes. Taking the convenience of road network traffic as an example, first, the reward network R uses a spatially aware enhanced graph convolutional neural network to encode the input road network topology, and uses the distance between nodes as a feature in the encoding process to capture the topological information of the road network graph structure and the spatial distance information between nodes. The encoding process can be expressed as Formula 2, where c i and c j They are node v i and v j The coordinate vector in the image coordinate system. j Each node v j The feature vector of is composed of the normalized coordinates and average direction angle of its neighbor nodes. i All neighbors of v j , this method aggregates the features of these nodes into To construct graph structure embedding vector where u ij represents the learnable weights.

[0072]

[0073] These encoded graph structures are embedded in vectors After being processed by a multi-layer perceptron and ReLU activation function, the topological attribute (i.e., traffic convenience) score corresponding to the graph structure will be calculated. This score will be used as one of the supervisory signals of the generator to control the generator to generate a road network with the expected properties (i.e., high traffic convenience).

[0074] The optimization goal of the entire road network generation process is composed of the WGAN-GP loss function related to the spatial position of the road network and the topological attribute reward score, so as to simultaneously model and learn the spatial position information and topological features of the road network in the training data, and then generate a large number of topological road network models with strong connectivity, satisfying the given spatial position constraints, and satisfying the expected attributes. As shown in Formula 3, x is the generated image, E(x) is the corresponding road network topology structure output by the differentiable topology extractor, and L w is the WGAN-GP loss function related to the spatial location of the road network, L r is the topological attribute reward loss function, and λ = 0.3 is used to balance the strength between the two.

[0075] L(x)=(1-λ)L w (x)+λL r (x) = (1-λ)L w (x)+λ(1-score(E(x)))(3)

[0076] Among them, the WGAN-GP loss can be expressed as Formula 4, where P G and P data They represent the data distribution generated by the generator and the distribution of real data respectively. is the sampling from these distributions, and β is the coefficient of the gradient penalty. The constant k is a constraint on the discriminator gradient, requiring that the gradient norm during the learning process does not exceed a certain constant k to improve the stability of the training process and alleviate the mode collapse problem.

[0077]

[0078] like Figure 4 As shown, a controllable generation system of a topological road network for autonomous driving simulation includes a data acquisition and processing module, a spatial position condition encoding module, a topological road network generation module, a road network format conversion module, and a road network quantitative evaluation module.

[0079] The data acquisition and processing module is used to collect and process road network data in the real world to form a data set for controllable generation and training of topological road networks.

[0080] Specifically, first, the road network data in the real world is collected from online map websites, and then rasterized into black and white binary images. The method steps for rasterizing the road network data in the real world are as follows: first, the geographic information coordinate system is selected as WGS84 (EPSG:4326), and the image scaling is determined (the actual distance represented by one pixel in the image is 10 meters), and then the road network segments in the real world are expanded into different road widths according to the road network level (10 pixels for main roads, 5 pixels for secondary roads, and 2 pixels for roads inside residential areas). In the corresponding rasterized image, the road network part is white and the background part is black.

[0081] Then, the population density map, land use type map, and river water area map of the area corresponding to the road network are collected simultaneously to form a spatial location constraint map.

[0082] Then, the rasterized image is divided into a number of image blocks. During the division process, the overlap ratio between the image blocks is 30%.

[0083] Then, for each image block, the black and white binary image is processed into a graph structure using a morphological method, and redundant nodes are removed and simplified into a road network topology structure. Specifically, first, node mapping is performed on the black and white binary image to distinguish the black background, the white road network, and the road network nodes. Then, the adjacent nodes are formed into a node domain, and each node domain is indexed and encoded. Secondly, starting from the node, the lines of the node neighborhood are traversed. If a node is found twice in the neighborhood, the line is used as the connection between the two nodes. Finally, the center point of the node domain is found, and all nodes and the connection relationship between them are output. For these nodes and connection relationships, the present invention simplifies all nodes with a degree of 2 and removes redundant nodes. Specifically, if the angle between the two edges associated with a node with a degree of 2 is greater than a set threshold (160 degrees in this embodiment), the redundant node is removed.

[0084] Then, the topological properties of the road network topology are calculated. Taking the road network traffic convenience as an example, the calculation method is to calculate all the node pairs v in the road network topology with a distance of more than 300 meters. i and v j , calculate d E (v i ,v j ) / d S (v i ,v j ), where d E and d S They are node v i and v j The Euclidean distance and the Dijkstra shortest distance between them.

[0085] Finally, the road network images, the corresponding spatial position constraint graphs, the corresponding topological structures, and the corresponding topological attributes are combined into a data set, and it is checked whether the number is greater than 20,000 for the training of the controllable generation model of the topological road network.

[0086] The spatial position condition encoding module encodes the collected spatial position constraint graph to obtain a feature graph of the spatial position constraint condition.

[0087] Specifically, during the model training process, the encoding module in the autoencoder is first used to encode the spatial position constraint map S into a feature map with a lower dimension, and then the decoding module is used to restore the encoded feature map to the spatial position constraint map S′ input to the encoder, and the highest similarity between S and S′ is used as the optimization goal. During the model inference process, only the encoding module in the autoencoder is used to process the feature map of the spatial position constraint condition.

[0088] The topological road network generation module takes the feature map of the spatial position constraint as input, and generates a road network image under the constraint of the feature map through the generator. The generated road network image and the feature map of the spatial position constraint are simultaneously sent to the discriminator to determine the authenticity of the road network image generated by the generator and whether it complies with the constraints of the spatial position feature map. In addition, the generated road network image will also be sent to the differentiable topological structure extractor, with the purpose of extracting the topological structure of the road network generated by the generator, so that the topological attribute reward network can calculate the topological attribute score of the generated road network, and the topological attribute score is used to guide the generator to generate a topological road network that meets specific topological attributes. The partial topological road network generated in this embodiment and the intermediate results (road network image) in the generation process are shown in the figure. Figure 5 shown.

[0089] The road network format conversion module divides the generated road network topology into sub-graphs, numbers each road in the road network, and adds virtual geographic information to convert it into the road network format required by the autonomous driving simulation environment.

[0090] Specifically, first, the road network topology is divided into subgraphs so as to number each road in the road network. The present invention stipulates that the subgraph must meet the following three properties: first, the subgraph is an acyclic graph; second, the degree of each node in the subgraph does not exceed 2; third, the angle formed by the two edges associated with any node in the subgraph is not less than a threshold (120 degrees in this embodiment). Then, the road network topology data is divided into several subgraphs by breadth-first search, and numbered to distinguish each road.

[0091] Then, according to the road number and road attributes (main road, secondary road or internal road in residential area, etc.), road elements and road width are generated. This embodiment refers to the OSM road specification (XML format specification) to generate four types of elements, including nodes (Node) representing geographical locations, channels (Way) representing linear elements or planar elements, relationships (Relation) between existing nodes and channels, and tags (Tag) for storing metadata of map elements. In the process of generating these elements, except for the nodes, the content of other elements can be empty, but the element field index must exist. Specifically, the present invention first stores the road topological structure after numbering and grouping as data of a CSV file. Then, the method defines a basic XML format template and adds XML characters representing the generated nodes and channels to it. Then, the present invention extracts the identification, longitude and latitude information of each road node in turn, generates XML characters for each node, and generates XML text of the road according to the road grouping number information.

[0092] Finally, convert the generated XML format file into a file that complies with the specifications of the autonomous driving simulation environment, and check its legality. Specifically, this method specifies the origin of the coordinate system for the generated road network as (latitude 30.25937, longitude 120.123115), and then converts the XML file into the corresponding format file based on the script provided by the autonomous driving simulation software CARLA, and checks the legality of the conversion process. Similarly, in order to meet the requirements of the autonomous driving road network simulation software SUMO, this method converts it into a traffic network description file and a traffic flow simulation model file. It is worth noting that the origin of the coordinate system can be selected arbitrarily, and this embodiment only provides one of the implementation methods. The rendering effect diagram of the autonomous driving simulation environment of the partial topology road network generated by this embodiment is shown in the following figure. Figure 6 shown.

[0093] The road network quantitative evaluation module calculates the traffic performance index (including road connectivity, traffic convenience, generation quality, and generation diversity) of the generated topological road network to verify the rationality of the generated topological road network model and the effectiveness of the topological road network generation method proposed in the present invention, that is, to verify that the topological road network generation method proposed in the present invention has the ability to generate a road network with strong connectivity, satisfying the specified spatial constraints, and having the expected topological attributes. Among them, the calculation method of road connectivity is the average value of the degree of all nodes in the road network topological structure. The higher the value, the stronger the connectivity; the calculation method of traffic convenience has been described in the previous text of this embodiment. The higher the value, the better the traffic convenience; the calculation method of generation quality is the FID index, which can be referred to Heusel, Martin, et al. "GANs Trained by a Two Time-Scale Update RuleConverge to a Nash Equilibrium." (2017). The lower the value, the better the generation quality; the generation diversity is used to measure the difference between the generated topological road network and the road network in the data set. The higher the value, the richer the generation diversity. This value is achieved by calculating the average value of the minimum Chamfer distance between all generated data and the sampled true value. Table 1 shows the comparison of the road network generated according to this embodiment and the road network generated by the prior art in the above quantitative evaluation indicators.

[0094] Table 1

[0095] Road connectivity Transportation convenience Build quality Generate diversity Prior art 2.29 76.72％ 17.25 61.47 This embodiment 2.68 85.43％ 9.79 62.51

[0096] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A controllable generation method of topological road network for autonomous driving simulation, characterized in that The steps include: Step S1: Encode the spatial position constraint condition to obtain an encoded spatial position feature map; Step S2: The generator uses the spatial position feature map as a constraint condition to generate a road network image that satisfies the spatial position constraint; Step S3: The discriminator determines the authenticity of the generated road network image and whether it conforms to the constraints of the spatial position feature map; Step S4: inputting the generated road network image into a differentiable topology structure extractor at the same time to extract the topology structure of the road network image end-to-end; The differentiable topological structure extractor includes an encoding module, a decoding module, a node coordinate detection head and a connection relationship detection head. The decoding module takes the encoding feature map output by the encoding module, a plurality of node coordinate marking blocks and a connection relationship marking block as input. The decoded node coordinate marking blocks and the connection relationship marking blocks are simultaneously sent to the node coordinate detection head and the connection relationship detection head. The node coordinate detection head is composed of a plurality of groups of multi-layer perceptrons to predict the coordinates between these nodes. The connection relationship detection head is also composed of a plurality of groups of multi-layer perceptrons. Each group of multi-layer perceptrons takes any two decoded node coordinate marking blocks and one connection relationship marking block as input to predict the connection relationship between these two nodes. The encoding module and the decoding module are composed of a Transformer-based deformable attention module, so that the query operation in the Transformer focuses on a small range of spatial features determined by a learnable offset near a reference point; Step S5: inputting the road network topology structure into the topology attribute reward network, calculating the topology attribute score of the road network topology structure, so as to guide the generator to generate a road network that satisfies specific topology attributes; The extracted road network topology is input into the topological attribute reward network. The reward network first encodes the road network topology and converts it into a graph structure embedding vector. The embedding vector is then sent to a regression module consisting of a multi-layer perceptron and an activation function to calculate the topological attribute score of the road network topology. Based on the attribute score, the road network generator is controlled to generate a road network that meets specific topological attributes. The reward network uses a spatially-aware enhanced graph convolutional neural network to encode the input road network topology structure, and uses the distance between nodes as a feature in the encoding process to capture the topological information of the road network graph structure and the spatial distance information between nodes; The optimization objective of the entire road network generation process is composed of a loss function related to the spatial position of the road network and a topological attribute reward score, so as to simultaneously model and learn the road network spatial position information and topological features in the training data, and then generate a topological road network model.

2. The method for controllable generation of a topological road network for autonomous driving simulation according to claim 1, characterized in that: In step S1, an autoencoder is constructed to obtain a spatial position constraint map S and encode it into a low-dimensional feature map; the feature map is then decoded to restore it to a spatial position constraint map S’ , based on the spatial position constraint graph S and the spatial position constraint graph S’ , train the autoencoder.

3. The method for controllable generation of a topological road network for autonomous driving simulation according to claim 1, characterized in that: In the step S2, a road network generator is constructed, a spatial position feature map is obtained, and the map is converted into a multi-channel marker block and a learnable position encoding marker block through a multi-layer perceptron. The marker block is sent to a plurality of Transformer modules and upsampling modules to gradually generate a road network image that satisfies the spatial position constraints; wherein the Transformer module is used to model the local detail features and global consistency of the image, and the upsampling module is used to increase the resolution of the image.

4. The method for controllable generation of a topological road network for autonomous driving simulation according to claim 3, characterized in that: In the process of gradually generating images, for the intermediate images whose size exceeds the threshold, they are divided into several grids, and then each grid block is regarded as a labeled block; The upsampling module uses a bicubic interpolation method to expand the image resolution of the low-resolution intermediate image and keep the number of image channels unchanged; for the high-resolution intermediate image, a pixel puzzle method is used to expand the image resolution and reduce the number of image channels; finally, after the road network image generated by multiple Transformer modules and upsampling modules is converted into a road network image of corresponding size using a linear projection layer.

5. The method for controllable generation of a topological road network for autonomous driving simulation according to claim 1, characterized in that: In the step S3, a road network discriminator is constructed, including a group of Transformer modules and a detection head. The generated road network image is divided into multiple image marker blocks of different scales through a linear embedding layer and sent to the corresponding Transformer modules of multiple stages respectively. The detection head includes a multi-layer perceptron, which makes a binary classification of whether the generated image is true or false. If the generated road network image is consistent with the distribution of the real road network data and meets the constraints of the spatial position feature map, it is judged to be true, otherwise it is false.

6. The method for controllable generation of a topological road network for autonomous driving simulation according to claim 5, characterized in that: The road network image is divided into three image marker blocks of different scales and sent to the three stages of Transformer modules respectively. The Transformer module of the first stage takes the image marker block of the first scale as input, sends it to the downsampling module after processing, and outputs a marker block of the second scale; then, the marker block is connected with the input image marker block of the same scale and sent to the Transformer module of the second stage; similarly, the image marker block of the third scale is obtained and sent to the third stage; after the third stage, the scale of the output marker block is kept unchanged, and a classification marker block is attached to assist the detection head in carrying out the classification task.

7. A system based on the controllable generation method of a topological road network for autonomous driving simulation according to claim 1, comprising a data acquisition and processing module, a spatial position condition encoding module, a topological road network generation module, a road network format conversion module, and a road network quantitative evaluation module, characterized in that: The data acquisition and processing module collects the road network data in the real world and rasterizes it into a black and white binary image, and collects the corresponding spatial position constraint data at the same time, then divides the rasterized road network image into a number of image blocks, processes them into a graph structure, calculates their topological properties, and forms a road network data set; The spatial position condition encoding module encodes the collected spatial position constraint data to obtain an encoded spatial position feature map; The topological road network generation module uses the spatial position feature map as a constraint condition to generate a topological road network that meets the spatial position condition constraints and has specific topological attributes; The road network format conversion module divides the generated road network topology into sub-graphs, and numbers each road in the road network and adds virtual geographic information to convert it into a road network format that meets the requirements of the autonomous driving simulation environment; The road network quantitative evaluation module calculates traffic performance indicators for the generated topological road network to verify the rationality of the generated topological road network model and the effectiveness of the topological road network generation method.

Images