Three-dimensional scene controllable generation method and system for automatic driving simulation

Abstract

The invention provides a three-dimensional scene controllable generation method and system for automatic driving simulation, and belongs to the technical field of computers. The method comprises the following steps: acquiring a road network topological graph to be subjected to three-dimensional scene generation, and constructing a geometric minimum ring set based on the road network topological graph; taking the geometric minimum ring as a boundary of a sub-region of a scene plane corresponding to the road network topological graph, and obtaining graph structure data of each sub-region; converting the graph structure data into a two-dimensional layout of the sub-regions, and then obtaining a scene plane layout; constructing a scene information table based on the scene plane layout, and obtaining a 3D model library; traversing the planar graph contour of each building in the scene information table, and screening three-dimensional models with contour matching degrees higher than a threshold value from the 3D model library to construct a candidate set; and performing optimization matching and replacement of the three-dimensional model on the building in the scene plane layout to obtain a three-dimensional scene. The diversity and visual quality of the generated three-dimensional scene are improved.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology and autonomous driving, and specifically relates to a controllable generation method and system of a three-dimensional scene for autonomous driving simulation. Background Art

[0002] Scene generation technology provides a large number of three-dimensional scenes for sensor perception, which provides support for exploring the impact of scene elements on the performance of perception algorithms and is of great significance for model training and perception algorithm optimization. In a multi-vehicle autonomous driving system, intelligent vehicles often perceive the driving environment based on sensors and process and analyze the perception results based on artificial intelligence algorithms. Autonomous driving algorithms based on artificial intelligence require a large amount of data for training and testing. Furthermore, researchers often need to answer the question "what kind of scenes are our autonomous driving algorithms suitable for?" Building simulation scenes in a virtual world is a very promising solution. However, most existing methods still rely on manual construction, and the number and diversity of scenes are limited. Therefore, this paper studies the automatic generation of a large number of three-dimensional scene models with different characteristics for sensor perception, which has important practical significance for autonomous driving perception model training, mutual adaptation sensitivity of scene elements and perception algorithms, and perception algorithm optimization.

[0003] In recent years, researchers have begun to design generative artificial intelligence to synthesize a large number of scenes. However, existing image-based methods learn pixel-level relationships in building layout data instead of modeling the building as a whole, and the generated results often have relatively fuzzy boundaries. Research based on vector graphics directly models building instances to directly generate vector building outlines. However, such methods are often based on rectangular boundaries and cannot be applied to scene boundaries of arbitrary shapes divided by road networks in actual situations. In addition, the results generated by these methods are single and rectangular, and their diversity is insufficient. Therefore, this paper models the building outline into multiple shapes and proposes a conditional variational learning model based on graph dynamic attention to achieve conditional generation based on arbitrary boundaries and given overall layout attributes. In response to the problem of inconsistent synthetic images in rendering-based 3D scene generation methods, this paper proposes an asset matching strategy to directly generate 3D city scene models, and then generate a large number of diverse 3D scenes for sensor perception for autonomous driving simulation. Summary of the invention

[0004] In order to solve the problems in the prior art, the present invention proposes a controllable generation method and system of three-dimensional scenes for autonomous driving simulation.

[0005] The technical solution adopted by the present invention is as follows:

[0006] In a first aspect, the present invention discloses a controllable generation method of a three-dimensional scene for autonomous driving simulation, comprising:

[0007] 1) Obtaining a road network topology map for generating a three-dimensional scene, and constructing a geometric minimum ring set based on the road network topology map;

[0008] 2) Divide the scene plane corresponding to the road network topology map using the geometric minimum ring as the boundary, obtain multiple sub-regions and assign attribute features to each sub-region, and then generate a binary image corresponding to each sub-region; obtain the trained convolution encoder and use it to extract the encoding vector of the binary image;

[0009] 3) Obtain the trained graph dynamic attention decoder and use it to decode the attribute features, encoding vectors, and hidden vectors obtained by sampling the standard normal distribution to obtain the graph structure data of each sub-region;

[0010] 4) Convert the graph structure data into a two-dimensional layout, and stitch all the two-dimensional layouts to obtain the scene plane layout;

[0011] 5) Build a scene information table based on the scene plane layout and obtain a 3D model library; traverse the floor plan outlines of each building in the scene information table, and select a candidate set of three-dimensional models with outline matching degrees higher than a threshold from the 3D model library;

[0012] 6) Optimize the matching based on the positional relationship and feature similarity of the buildings in the scene plane layout, and finally perform a 3D model replacement operation to generate the final 3D scene.

[0013] In a second aspect, the present invention discloses a controllable three-dimensional scene generation system for autonomous driving simulation for implementing the method, comprising:

[0014] A coding vector and attribute feature acquisition module, which is used to obtain a road network topology map to be used for three-dimensional scene generation, and to construct a geometric minimum ring set based on the road network topology map;

[0015] Divide the scene plane corresponding to the road network topology map using the geometric minimum ring as the boundary, obtain multiple sub-areas and assign attribute features to each sub-area, and then generate a binary image corresponding to each sub-area; obtain the trained convolution encoder and use it to extract the encoding vector of the binary image;

[0016] The graph structure data acquisition module is used to obtain the trained graph dynamic attention decoder and use it to decode the attribute features, encoding vectors and hidden vectors obtained by sampling the standard normal distribution to obtain the graph structure data of each sub-region; the scene plane layout acquisition module is used to convert the graph structure data into a two-dimensional layout and splice all the two-dimensional layouts to obtain the scene plane layout;

[0017] A candidate set acquisition module is used to construct a scene information table based on the scene plane layout and obtain a 3D model library; traverse the floor plan outlines of each building in the scene information table, and select three-dimensional models with outline matching degrees higher than a threshold from the 3D model library to construct a candidate set;

[0018] The 3D scene generation module is used to optimize the matching based on the position relationship and feature similarity of the buildings in the scene plane layout, and finally perform the 3D model replacement operation to generate the final 3D scene.

[0019] Compared with the prior art, the present invention has the following beneficial technical effects:

[0020] (1) The present invention proposes a controllable generation method for three-dimensional scenes for autonomous driving simulation, aiming to controllably generate a large number of three-dimensional scenes with different features that can be sensed by sensors, and apply it to autonomous driving simulation tests. In view of the problem that the existing technology has weak diversity and poor visual quality of the generated results, the present invention proposes a graph dynamic attention encoder-decoder trained based on conditional variational learning, and uses the trained graph dynamic attention decoder to decode the attribute features and encoding vectors obtained based on the road network topology map to be generated for the three-dimensional scene, as well as the hidden layer vectors obtained by sampling the standard normal distribution, to obtain graph structure data, thereby improving the diversity and visual quality of the generated three-dimensional scene results. In view of the problem that the existing technology is difficult to controllably generate based on given boundaries and attributes, when training the graph dynamic attention encoder-decoder, the present invention introduces virtual nodes into the graph structure data input to the graph dynamic attention encoder to enhance the strength of the attribute features in the conditional control, so that the generated scene can meet the constraints of the input boundaries and scene attributes. In addition, the present invention proposes a scene asset matching scheme based on the CLIP model, that is, when optimizing the matching between the building and the three-dimensional model, the CLIP image encoder is used to obtain the material features of the three-dimensional model to generate a three-dimensional scene that can be sensed by the sensor.

[0021] (2) The present invention proposes a controllable three-dimensional scene generation system for autonomous driving simulation. The scene plane layout data acquisition and processing module included in the system can efficiently and comprehensively collect the training data required by the controllable scene plane layout generation method; the scene quality evaluation module included in the system can simultaneously evaluate the quality of the generated three-dimensional scene in a qualitative and quantitative manner, thereby verifying the effectiveness of the controllable three-dimensional scene generation method. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a structural diagram of the controllable three-dimensional scene generation system for autonomous driving simulation proposed in the present invention.

[0023] Figure 2It is a structural diagram of a convolution-based codec and a generation result diagram of the codec of the method for locally controllable generation of a three-dimensional scene proposed in the present invention.

[0024] Figure 3 It is a training flow chart of the graph dynamic attention codec of the three-dimensional scene local controllable generation method proposed in the present invention.

[0025] Figure 4 It is a schematic diagram of the process of adding virtual nodes to the dynamic attention encoder in the controllable three-dimensional scene generation method proposed in the present invention.

[0026] Figure 5 It is a flow chart of matching buildings and three-dimensional models in the controllable generation method of three-dimensional scenes proposed by the present invention.

[0027] Figure 6 It is a flow chart of CLIP-based 3D model optimization matching in the 3D scene generation method proposed in the present invention.

[0028] Figure 7 It is a scene plane layout result diagram obtained by the scene plane layout generation method proposed in the present invention and a comparison diagram with the results of the existing pix2pix model.

[0029] Figure 8 It is a three-dimensional scene result diagram obtained by the three-dimensional scene generation method proposed in the present invention and a comparison diagram with the existing SGAM model results. DETAILED DESCRIPTION

[0030] The present invention is further described and illustrated below in conjunction with specific embodiments. The embodiments are merely exemplary of the present disclosure and do not define the scope of limitation. The technical features of each embodiment of the present invention may be combined accordingly without conflicting with each other.

[0031] Based on a generative artificial intelligence model, the present invention proposes a controllable generation method and system of three-dimensional scenes for autonomous driving simulation, aiming to controllably generate a large number of three-dimensional scene models with different features that can be used for sensor perception, and apply them to autonomous driving simulation tests.

[0032] In a first aspect, the present invention provides a controllable generation method of a three-dimensional scene for autonomous driving simulation, comprising the following steps:

[0033] S1: Obtain a road network topology map for three-dimensional scene generation, and construct a geometric minimum ring set based on the road network topology map.

[0034] The process of finding the plane layout of the scene surrounded by the road network layout can be modeled as the problem of searching for a geometric minimum loop from a road network topology graph. The present invention defines a geometric minimum loop as a minimum geometric closed figure that does not contain any ring structure under the constraints of a plane coordinate system. The present invention proposes a geometric minimum loop search method, which analyzes simple paths from a random node to all its adjacent nodes, and iteratively deletes the remaining shortest cyclic paths based on these paths.

[0035] When obtaining the geometric minimum loop, first obtain the road network topology map to be used for three-dimensional scene generation, wherein the road network topology map includes nodes and edges, nodes represent intersections or road bends, edges represent roads between two nodes (i.e., vehicle-accessible areas), and node features of nodes include the location of nodes (represented by horizontal and vertical coordinates). Then remove all nodes with a degree of 1 in the road network topology map (these nodes cannot form a loop and belong to isolated chains) to simplify the road network topology map. Next, traverse all remaining nodes v with a degree of 2 in the road network topology map, and perform the following operations on each node v: find its two neighbor nodes v1 and v2, and search for all simple paths from node v to neighbor node v1 (i.e., paths without repeated nodes). Then, check whether these paths contain another neighbor node v2. If there is a path containing neighbor node v2, connect this path with the current node v to form a loop, and find the shortest loop among them. Then, the shortest loop is taken as the geometric minimum loop, and the current node v is removed from the road network topology to avoid repeated calculations. Then all nodes with degree 1 in the current road network topology are removed, and then all nodes with degree 2 remaining in the road network topology are traversed to find the geometric minimum loop, and then the current node is removed from the road network topology. Finally, the above steps are repeated until there are no more nodes remaining in the road network topology. At this time, all the geometric minimum loops obtained constitute the geometric minimum loop set C.

[0036] S2: Divide the scene plane corresponding to the road network topology map with the geometric minimum ring as the boundary, and construct a binary image based on the geometric minimum ring, then obtain the trained convolution encoder, use the convolution encoder to encode the binary image, and obtain the boundary coding vector; and assign attribute features to each sub-area. Among them, the boundary coding vector is the coding vector; the attribute features include the density of the building complex and the height of the building complex in the sub-area. The density of the building complex is divided into high density and low density. When the density of the building complex in the sub-area is greater than or equal to 0.1 (the proportion of the building complex area to the sub-area area), the density of the building complex is high density, otherwise it is low density; the height is divided into high-rise and low-rise. When the average height of the building complex is greater than or equal to 20m, it is high-rise, otherwise it is low-rise.

[0037] In a specific embodiment of the present invention, the scene plane corresponding to the road network topology diagram is divided into sub-areas using a geometric minimum ring, and the geometric minimum ring is used as the boundary of the sub-area of ​​the scene plane layout.

[0038] Step S2 uses the trained convolutional codec to model the boundaries of the sub-regions of the scene plane and obtain boundary coding vectors to achieve the generation of the building plane layout under the subsequent specified boundary conditions. Figure 2 As shown in the figure, the core structure of the convolutional encoder and decoder consists of a convolutional encoder and convolutional decoder It consists of two parts; the convolution encoder consists of 4 convolution layers, and the convolution decoder is the mirror image of the convolution encoder. When obtaining the boundary coding vector, first obtain the coordinates of each node of the geometric minimum ring, and then create a blank image of the same size as the scene plane, and the initial pixel values ​​are all 0. Then draw the geometric minimum ring in the image based on the node coordinates of the geometric minimum ring, and set the pixel value of the area within the geometric minimum ring to 1, and keep the pixel value outside the geometric minimum ring to 0, and finally generate a scene plane binary plane boundary image with the area within the geometric minimum ring as 1 and the area outside the ring as 0. The scene plane binary plane boundary image is segmented based on the boundary defined by the geometric minimum ring to obtain the binary image corresponding to each geometric minimum ring itself. At the same time, when the binary image is input into the trained convolution encoder, it is necessary to unify the size of the binary image. For example, in a specific embodiment of the present invention, the size of the binary image can be 256×256. The trained convolution encoder compresses the input binary image into a boundary coding vector During training, the convolution decoder uses the boundary encoding vector compressed by the convolution encoder As input, the restored binary image is output. The L1 distance between the binary image restored by the convolution decoder and the original binary image is used as the loss function to update the parameters of the convolution encoder and decoder to achieve self-supervised training.

[0039] Before training the convolutional codec, it is necessary to prepare the training set first. In this embodiment, the road network topology map and the corresponding scene plane layout vector map in the real scene are collected from online websites. Then, based on the road network topology map, the scene plane corresponding to the scene plane layout vector map is divided into several sub-areas in this embodiment. Specifically, the scene plane is divided into several sub-areas according to the geometric minimum ring search method proposed in step S1 of the present invention. Then, in this embodiment, the scene plane layout vector map of the sub-area is rasterized into image data. Among them, the method steps for rasterizing the scene plane layout of the sub-area are specifically as follows: first, a geographic information coordinate system is selected, and the scaling scale of the image corresponding to the scene plane layout vector map of the sub-area (i.e., the actual distance represented by one pixel in the image) is determined, and then the internal value of the building polygon in the scene plane layout vector map of the sub-area is set to the building height, and the remaining area is set to 0. Then, this embodiment completes the missing building height information in the collected data. For the building layout sample, this embodiment observes that the building outline extracted from the online website often lacks height information, and the building sample is partially missing. Therefore, the present invention combines it with a public data set containing building height information.

[0040] Specifically, for the Chinese region, this embodiment uses the height information in the CNBH dataset [WU WB, MA J, BANZHAF E, et al. A first Chinese building height estimate at 10 m resolution (CNBH-10m) using multi-source earth observations and machine learning [J]. RemoteSensing of Environment, 2023, 291: 113578-113591.]; for foreign regions such as the United States, Australia, and Europe, this embodiment uses the dataset provided by Microsoft [https: / / github.com / microsoft / GlobalMLBuildingFootprints]. The key to the processing process is the data alignment method and data merging rules. Therefore, in this embodiment, the coordinate reference system of all building outline spaces is standardized as the pseudo-Mercator projection (EPSG:3857), so that the data from three different sources are aligned in the spatial coordinates. Then, the height information is assigned to each building in the original scene plane. When merging two sets of building outlines, taking the CNBH dataset as an example, this paper follows the following rules: Taking OSM (OpenStreetMap, Open Street Map) data as a benchmark, index the corresponding area in the CNBH dataset, calculate the average height value of the effective pixels, and finally assign the average height value to the corresponding building as height information. Specifically: For areas where the building height value is missing in the sub-area, this embodiment uses nearby buildings within a radius of 300 meters to estimate the height to update the scene plane layout vector map of the sub-area. Finally, the sub-areas of all scene planes and the scene plane layout vector map of the updated sub-areas constitute a data set, which is used as the training data of the graph dynamic attention codec for obtaining the trained graph dynamic attention decoder. Finally, this embodiment calculates the average height of the buildings in the sub-area and the density of the building group, and then assigns attribute features to each sub-area based on the calculation results. The attribute features include the density of the building group and the height of the building group. The density of the building group is divided into high density and low density, and the height of the building group is divided into high-rise and low-rise. Finally, the division of the scene plane sub-area is completed and the corresponding attribute features are assigned to each sub-area.

[0041] When training the convolutional encoder-decoder, as shown in the attached Figure 2As shown, each sub-region in the data set is converted into a binary image, and the sizes of the converted binary images are unified, and then all the unified binary images constitute a training set for training the convolutional codec, so as to implement the training of the convolutional codec, obtain the trained convolutional codec, and then take out the trained convolutional encoder to perform the operation of step S2.

[0042] S3: Obtain a trained graph dynamic attention decoder, use the attribute features and boundary encoding vectors of the sub-region and the hidden layer vector obtained by sampling the standard normal distribution as inputs of the graph dynamic attention decoder, and obtain the graph structure data of each sub-region.

[0043] Specifically, step S3 uses the boundary encoding vector of the sub-region corresponding to the geometric minimum ring obtained in step S2, the attribute features of the sub-region in step S2, and the hidden layer vector obtained by sampling the standard normal distribution as the input of the trained graph dynamic attention decoder to obtain the graph structure data of each sub-region, that is, to generate the scene plane layout result of the buildings in each sub-region.

[0044] The trained graph dynamic attention decoder is obtained based on training the graph dynamic attention codec. When training the graph dynamic attention codec, the training data can be prepared first, that is, during training, the data set for training the graph dynamic attention codec is prepared according to the above steps, wherein the scene plane layout vector diagram of the sub-region is the building layout vector diagram inside the sub-region. Then, the scene plane layout vector diagram of each sub-region in the data set is standardized and converted into graph structure data.

[0045] In order to convert the scene plane layout into structured data, that is, into graph structure data, and facilitate the processing and learning of the deep learning model, the present invention standardizes the expression of the building monomers in the sub-area and the overall layout of the sub-area respectively. Combined with the sample distribution in the building layout data set used in this embodiment, the present invention divides the existing buildings into six categories: rectangular, "X"-shaped, "L"-shaped, "I"-shaped, "U"-shaped and elliptical. The key to the standardized expression of the existing building monomers is to determine the type of building monomers and solve the corresponding parameter set. Since this problem is an unconstrained nonlinear optimization problem and the derivatives are difficult to calculate during the solution process, this embodiment adopts Powell's method [VASSILIADIS VS, CONEJEROS R. Powell methodPowell Method [M] / / FLOUDAS CA,PARDALOS P M. Encyclopedia of Optimization. Boston; Springer. 2001: 2001-2003.] to solve the problem and obtain the scene plane layout graph structure data, that is, the graph structure data. The nodes in the graph structure data represent the normalized building monomers, and the edges represent the connection relationship between the nodes, that is, the relative position relationship between the building monomers.

[0046] Then we construct a graph dynamic attention encoder, which is an important model for graph representation learning. Figure 3 As shown, this embodiment introduces a learnable dynamic attention mechanism into the graph neural network to form the graph dynamic attention encoder to improve the encoding performance of scene planar graph structure data.

[0047] In the controllable generation task of the scene plane layout of the present invention, the overall properties of the scene plane layout are the focus of attention. Although the graph dynamic attention encoder enhances the expressive power of the model, it does not change the local features of the original graph neural network that only focuses on neighbor nodes. Therefore, the graph dynamic attention encoder still does not pay enough attention to the overall features of the graph. Therefore, this embodiment adds virtual nodes to the scene plane layout graph structure data to strengthen the graph dynamic attention encoder's attention to the overall features of the scene plane layout graph structure data. Specifically, as shown in the attached Figure 4 As shown, this embodiment adds a virtual node connected to all nodes in the original scene plane layout graph structure data on the basis of the scene plane layout graph structure data obtained after conversion after normalized expression, and obtains the first graph structure data, so as to better retain the global structure and pay attention to the overall properties of the scene plane in the process of graph representation learning, so as to achieve better conditional control of the overall properties of the scene plane layout during the generation process.

[0048] This embodiment defines each node in the first graph structure data The node characteristics are , node features Represents the key attributes of a building unit. For example:

[0049]

[0050] in, Represents a node Whether there is actually a building unit at the location; Represents a node The type of building units located at the site; Represents a node The horizontal coordinate of the center position of the building unit at; Represents a node The vertical coordinate of the center position of the building unit; Represents a node The height of the building unit at the location; Represents a node The length of the minimum rotation bounding box of the floor plan outline of the building unit at the location; Represents a node The width of the minimum rotation bounding box of the floor plan outline of the building unit at the location; Represents a node The proportion of the floor plan outline of the building unit in the minimum rotation bounding box.

[0051] It has been observed experimentally that the calculation process of the attention weights of the graph neural network is static, that is, for different query nodes, the importance ranking of the nodes is fixed and has nothing to do with the query nodes. Therefore, this embodiment uses dynamic attention to improve the diversity of the results generated by the graph neural network. Specifically, in traditional graph neural networks, the calculation of attention weights depends on the score function obtained by sequentially operating the weight matrix and vector. During the training process, this operation method may cause the attention weights to collapse into a linear layer, thereby limiting the expressive power of the model. To solve this problem, this embodiment first converts the query node features (i.e., the node The node characteristics of ) and key node features (i.e. neighbor nodes The node characteristics of ) for concatenation, and then using a learnable weight matrix Perform a linear transformation on the concatenated result. Then use the LeakyReLU activation function on it and then add it to the attention parameter vector Multiply to calculate the correlation coefficient , the process is formally expressed as:

[0052] ;

[0053] in, is a learnable weight matrix that maps the original feature vector to a high-dimensional space; It is a node Neighboring nodes The correlation coefficient between nodes Neighboring nodes The influence coefficient of Represents a splicing operation; is the attention parameter vector; is the activation function; For Node The node characteristics of Neighbor Node The node features.

[0054] Get the correlation coefficient After that, the node features of each node are weighted and summed with those of its neighbor nodes, and then the nonlinear activation function is applied to obtain The final output feature vector. In order to enhance the model's expressiveness, this paper adopts a multi-head attention mechanism, that is, using multiple attention mechanisms to independently calculate the attention coefficients and concatenate or average their outputs. As shown in the formula:

[0055] ;

[0056] ;

[0057] in, Represents a node and neighbor nodes The weight coefficient between ; σ is the nonlinear activation function, is the number of attention groups, It is The first attention layer Nodes calculated by the group attention mechanism and neighbor nodes The weight coefficient between It is The first attention layer The input linear transformation matrix of the group attention mechanism, It is a node The The feature vector calculated by the attention layer, It will be input to the next attention layer for iterative calculation; For Node The set of neighbor nodes of ; For Node Neighbor nodes The The feature vector calculated by the attention layer, where .

[0058] In this embodiment, the feature vectors of each node are integrated into a feature matrix ,in Indicates the input The feature vector of the attention layer (i.e. The feature vector output by the layer), therefore, the above graph dynamic attention encoder can be formally described as:

[0059] ;

[0060] in, is the function of the graph dynamic attention encoder; For the The feature matrix of the layer.

[0061] In this embodiment, the feature matrix output by each layer is aggregated into an aggregate vector .

[0062] The graph dynamic attention decoder also uses the network model of the graph dynamic attention encoder as the backbone network, but it is a mirrored graph dynamic attention encoder. Specifically, the boundary encoding vector of the sub-region , attribute characteristics of sub-regions As the input of the graph dynamic attention decoder, and the aggregate vector Sampling is performed to obtain the hidden vector of the graph dynamic attention decoder , and the hidden vector It is also used as the input of the graph dynamic attention decoder. First, the graph dynamic attention decoder uses the hidden vector As input, the result is sent to the multi-layer perceptron to obtain the initial feature matrix , For Node The feature vector decoded by the 0th layer of the graph dynamic attention decoder; then message passing and feature learning are performed layer by layer. After the layer, the final feature matrix is ​​obtained , where each feature component After being decoded by different multi-layer perceptrons, the node features corresponding to each building monomer are obtained. ,in, Indicates the decoded node Whether there is actually a single building at the location; Indicates the decoded node The type of building units located at the site; Indicates the decoded node The horizontal coordinate of the center position of the building unit at; Indicates the decoded node The vertical coordinate of the center position of the building unit; Indicates the decoded node The height of the building unit at the location; Indicates the decoded node The length of the minimum rotation bounding box of the floor plan outline of the building unit at the location; Indicates the decoded node The width of the minimum rotation bounding box of the floor plan outline of the building unit at the location; Indicates the decoded node The proportion of the floor plan outline of the building unit in the minimum rotation bounding box.

[0063] This embodiment uses conditional variational learning to train the graph dynamic attention encoder-decoder. During the training process, conditional variational autoencoders are used for conditional variational learning. The goal of the conditional variational autoencoder is to learn and model the complex distribution of the floor plan layout of the buildings in the dataset. , which represents the distribution of the encoding vector at a given boundary and attribute characteristics Under the condition of This paper adopts the common method in conditional autoencoders for training, that is, the model models the distribution by maximizing its variational lower bound (ELBO) , the loss function of the training process is as follows:

[0064] ;

[0065] in, is the loss function; is a hyperparameter; is the encoding vector at the boundary (encoding vector) , attribute characteristics And the first graph structure data Under the condition of The expectation of the posterior distribution of ; Represents the vector encoded at the boundary , attribute characteristics And the first graph structure data Under the condition of The posterior distribution of ; that is, the distribution of the encoder output results. To encode the vector at the boundary , attribute characteristics and the hidden vector Under the condition of Distribution of is the hidden vector The prior distribution of is set as the standard normal distribution in this paper; for and The KL divergence distance between two distributions. is a hyperparameter; Represents the attribute att of the building in the graph structure data (i.e., the first graph structure data) input to the encoder (for example, the attribute att may be the height of the building); The attribute att of the building in the graph structure data (i.e., the second graph structure data) output by the decoder; is the L2 norm; this paper uses L2 loss to measure the attributes of each building in the graph structure restored by the decoder The properties of buildings in the real graph structure degree of similarity.

[0066] During the training process, this embodiment samples the aggregate vector output by the graph dynamic attention encoder and inputs the sampling result into the graph dynamic attention decoder, thereby training the graph dynamic attention encoder and decoder. After the training is completed, a trained graph dynamic attention encoder and decoder is obtained, and the graph dynamic attention decoder is used in the reasoning (generation) process of step S3.

[0067] S4: converting the graph structure data into a two-dimensional vector layout of the sub-region, and then splicing all the two-dimensional vector layouts to obtain a scene plane layout.

[0068] In step S4 of the present invention, the decoder output result (graph structure data) of step S3 is converted into the two-dimensional vector layout of the sub-region, which is the inverse process of step S3. Specifically, the nodes in the decoded graph structure data are restored one by one to the outline of the building monomer, and the node features are restored to the features of the building monomer outline. For example, one of the node features obtained by decoding The corresponding type in the building monomer type is restored, and then the two-dimensional vector layouts of all sub-areas are merged to form the scene plane layout of the entire scene. The result is the scene plane layout generated by the method provided in this embodiment.

[0069] S5: construct a scene information table based on the scene plane layout, and obtain a 3D model library; traverse the floor plan outline of each building in the scene information table, and select multiple three-dimensional models matching its outline from the 3D model library to form a candidate set of the building.

[0070] Based on the scene plane layout generated in step S4, this embodiment maintains a scene information table and a 3D model library. The scene information table maintained in this embodiment is composed of information such as the ID identification of the building unit, the ID of the sub-area of ​​the plot to which it belongs, the type of the building unit, the outline of the building unit plan, the height of the building unit, the area size, etc. In addition, this embodiment forms a 3D model library by collecting three-dimensional building unit models (i.e., three-dimensional models) of various shapes and materials from the Internet. The length, width, and height information of these three-dimensional building unit models can be edited. In order to more comprehensively extract the information and features of the three-dimensional building unit models, this embodiment takes snapshots of each three-dimensional building unit model in the 3D model library from different angles to obtain its key information. Specifically, as shown in the attached Figure 5 As shown, the present invention tilts the three-dimensional building monomer model from the front view Degree viewing angle (where Take snapshots from eight viewing angles (45, 90, 135, 180, 225, 270, 325, 360) and obtain the top view outline of each 3D building model. These snapshots and top view outlines are part of the 3D model library.

[0071] For the plane contour matching problem of building monomers, this embodiment selects building monomers with a high degree of matching with the plane contour of the building in the scene information table from the 3D model library as candidates, and all candidates constitute a candidate set. During the matching process, this embodiment solves the editable parameters of the three-dimensional building monomer model in the 3D model library (i.e., the length q1 and width q2 of the minimum bounding box of the plane contour when viewed from above) to adjust the corresponding parameters when the matching degree is maximized. Specifically, the contour matching problem is transformed into an optimization problem, which can be formally expressed as: given a polygonal contour with fixed parameters , that is, the floor plan outline of the building in the scene information table; there is another variable polygon outline , that is, the top view outline of the three-dimensional building model in the 3D model library, the length of the minimum outer bounding box of the outline and width are two adjustable parameters. The cost function is defined as the maximum intersection-over-union ratio between two polygonal contours. The specific calculation formula is:

[0072] ;

[0073] in, The top view of the 3D model The length of the minimum outer bounding box and width In the case of a building with a floor plan outline Top view of the 3D model The degree of matching; The floor plan outline of the building Top view of the 3D model The intersection and union ratio of The floor plan outline of the building Top view of the 3D model The area of ​​the intersection; Represents the floor plan outline of a building The area of Represents the top view of the 3D model The length of the minimum outer bounding box and width In the case of area.

[0074] To solve the optimization problem, this embodiment uses the Powell algorithm to find the parameter that minimizes the cost function through iterative search, that is, the length of the minimum outer bounding box of the three-dimensional building monomer model under the best contour matching. and width The intersection-over-combination ratio between the top view outline of the three-dimensional building model after the final iteration and the plan view outline of the building in the scene information table is , as the matching degree score. The present invention uses the matching degree score All three-dimensional building monomer models that are greater than a set threshold (in a specific embodiment of the present invention, the threshold is set to 0.8) are used as candidates, that is, as elements in the candidate set. For each three-dimensional building monomer model in the candidate set, the present invention sets another adjustable parameter (height parameter) as the value of the corresponding "height" attribute in the scene information table. After this step, the present embodiment obtains a candidate set of buildings in the scene plane layout, so the search space of the three-dimensional building monomer models corresponding to the buildings in the scene plane layout is compressed. Therefore, step S6 matches the asset material of the three-dimensional building monomer model in the pruned search space.

[0075] S6: Extract the features of each 3D model in the candidate set, optimize the matching of the 3D models based on the positional relationship and material similarity of the buildings in the scene plane layout, and then replace the buildings in the scene plane layout with the optimized 3D models to finally obtain the 3D scene.

[0076] The basis for matching the asset material of a three-dimensional building model is to extract the features of the asset material to achieve efficient feature matching. This embodiment uses the CLIP large model zero-sample learning capability to extract the material features of the three-dimensional building model, where the features are the materials of the three-dimensional building model, to achieve matching between asset materials. Specifically, as shown in the attached Figure 6As shown, each view image of the same three-dimensional building monomer model stored in the 3D model library is converted into a feature vector after being passed through an encoder. The present invention concatenates the feature vectors of each view image to obtain the encoding vector of the three-dimensional building monomer model, that is, to obtain the material characteristics of the three-dimensional building monomer model. In the process of asset material matching, for the encoding vector of the three-dimensional building monomer model of another building input, this embodiment calculates the cosine similarity between the two vectors.

[0077] Furthermore, compared to the plane contour matching in the previous step, which is for a single building, the asset material matching in this step is performed on the entire scene layout. Specifically, for each building in the scene layout , there exists a candidate set , the goal of this method is to select a single 3D building model from each candidate set to form the entire scene, so that the material characteristics of buildings in the same sub-area are as similar as possible, while the material characteristics of buildings in different sub-areas are as different as possible. This problem can be formally expressed as a binary linear programming problem: for a candidate set consisting of several candidate sets (in , is the total number of buildings in the scene plane), and the corresponding material feature vector set encoded by the CLIP model (in , Candidate Set A collection of material features for a single 3D building model. for The material features of the j-th three-dimensional building monomer model in the Each candidate set (also a set) such that under the constraints, the cost function Minimum.

[0078] ;

[0079] ;

[0080] ;

[0081] ;

[0082] in, is the kth building in the scene plane; is the number of 3D models in the candidate set of the kth building; To indicate whether the i-th 3D model in the candidate set of the k-th building is selected; Represents the material features of the i-th 3D model in the candidate set of the k-th building; Used to indicate buildings and buildings Whether they are in the same sub-area. A value of -1 indicates that they are in the same sub-area, and a value of 1 indicates that they are not in the same sub-area.

[0083] The first line of constraints means: if two buildings are located in the same sub-area of ​​the plot, the materials of their three-dimensional building models are made as similar as possible, that is, when the cosine similarity is the largest, the value of the cost function is the smallest; if they are located in different sub-areas, the materials of their three-dimensional building models are made as dissimilar as possible according to the distance, that is, when the cosine similarity is the smallest, the value of the cost function is the largest. The second line of constraints means: for the candidate set There are two situations for the three-dimensional building monomer model in the constraint condition: one is selected or not selected; the third line of the constraint condition means that two three-dimensional building monomer models cannot be assigned to the same building monomer at the same time.

[0084] This embodiment solves the binary linear programming problem with reference to the method of Bouzas et al. [BOUZAS V, LEDOUX H, NAN L. Structure-awareBuilding Mesh Polygonization [J]. ISPRS Journal of Photogrammetry and RemoteSensing, 2020, 167: 432-442.].

[0085] In the second aspect, corresponding to the controllable generation method of the three-dimensional scene provided above, the present invention also provides a controllable generation system of the three-dimensional scene for autonomous driving simulation. The structure of the controllable generation system of the three-dimensional scene for autonomous driving simulation proposed by the present invention is as follows: Figure 1 As shown, including:

[0086] The coding vector and attribute feature acquisition module M100 is used to obtain a road network topology map to be used for three-dimensional scene generation, and to construct a geometric minimum ring set based on the road network topology map;

[0087] The scene plane corresponding to the road network topology map is divided using the geometric minimum ring as the boundary to obtain multiple sub-areas, and a binary image corresponding to each sub-area is generated; the trained convolution encoder is obtained and used to extract the encoding vector of the binary image; and attribute features are then assigned to each sub-area;

[0088] The graph structure data acquisition module M200 is used to obtain the trained graph dynamic attention decoder, and use it to decode the attribute features, the encoding vector and the hidden vector obtained by sampling the standard normal distribution to obtain the graph structure data of each sub-region;

[0089] The scene plane layout acquisition module M300 is used to convert the graph structure data into a two-dimensional layout, and to splice all the two-dimensional layouts to obtain the scene plane layout;

[0090] The candidate set acquisition module M400 is used to construct a scene information table based on the scene plane layout and obtain a 3D model library; traverse the floor plan outlines of each building in the scene information table, and select three-dimensional models with outline matching degrees higher than a threshold from the 3D model library to construct a candidate set;

[0091] The three-dimensional scene generation module M500 is used to optimize the matching based on the position relationship and feature similarity of the buildings in the scene plane layout, and finally perform a three-dimensional model replacement operation to generate a final three-dimensional scene.

[0092] Furthermore, the three-dimensional scene controllable generation system also includes a scene plane layout data acquisition and processing module, a roadside element addition and scene export module and a generated scene quality evaluation module.

[0093] The scene plane layout data acquisition and processing module is used to collect and process scene plane layout data in the real world to form a data set for training the controllable generation of scene plane layout.

[0094] In addition to adding three-dimensional building models to the scene plan, the roadside element addition and scene export module in this embodiment uses a programmatic method to add trees, fences and other regularly distributed scene asset elements on the roadside at equal distances to increase the richness of the scene for perception by autonomous driving vehicles, and packages the entire generated three-dimensional scene and exports it to FBX format for use by autonomous driving simulation software such as CARLA.

[0095] In addition to qualitatively analyzing the effect of the generated scene quality evaluation module, this embodiment calculates the generation quality indicators (including visual quality, generation diversity, and key attribute distribution) of the generated three-dimensional scene to verify the rationality of the generated three-dimensional scene and the effectiveness of the three-dimensional scene generation method, that is, to verify that the three-dimensional scene generation method proposed in this embodiment is capable of generating three-dimensional scenes with high visual quality and strong diversity.

[0096] The evaluation method for generating a scene quality evaluation module comprises the following steps:

[0097] S701: In the CARLA autonomous driving simulation software, extract the roadside view image sequence of the generated 3D scene, and use the FID (Frechet Inception Distance) indicator to calculate its visual quality. Specifically, the indicator is calculated as follows: the smaller the FID value, the more similar the distribution of the generated data is to the real data, and the higher the visual quality.

[0098] ;

[0099] in, is the similarity between the generated image sequence and the real image sequence; X and Y represent the feature distribution of the generated results extracted by the Inception V3 model (derived from the extracted generated 3D scene roadside view images) and the real results (derived from the roadside view images in the real world), respectively; and represents the mean of the corresponding distribution; and represents the mean covariance of the corresponding distribution.

[0100] S702: Based on the two-dimensional projection of the building in the scene, the diversity of the generated results is calculated. Specifically, the present invention calculates the average intersection over union (mIoU) between the generated results and the true results to evaluate their diversity (denoted as DIV). The specific formula is:

[0101] ;

[0102] in, All the building floor plans in the collected dataset (real data); It is the two-dimensional plane projection of all buildings in the generated scene (generated result). The larger the value, the greater the diversity of the generated results.

[0103] S703: In order to further measure the difference in distribution between the generated results and the real data, the present invention observes two key attributes, namely, building density and building height, and measures the bulldozer distance Bden (WD) and Bhgt (WD) between the distribution of the buildings in the generated three-dimensional scene (generated results) and the buildings in the collected data set (real results) in terms of these two attributes.

[0104] S704: The present invention uses the validity index to evaluate whether the generated results meet the constraints of the input conditions. Specifically, the calculation method is to average the number of buildings that exceed the layout boundary limit in each generated sample.

[0105] First, this embodiment extracts the roadside perspective image sequence of the generated 3D scene and uses the FID (Frechet Inception Distance) indicator to calculate its visual quality. The scene quality evaluation results generated by this embodiment are shown in Table 1 (where ↓ means the lower the value, the better, ↑ means the higher the value, the better):

[0106] Table 1

[0107]

[0108] Table 1 shows the quantitative comparison between this embodiment and the prior art in terms of visual quality, diversity of generated results and other indicators. It can be seen from Table 1 that the visual quality indicators of the three-dimensional scene generated by this embodiment are numerically reduced by 64.61% compared with the prior art, indicating that the visual quality of the generated scene is greatly improved compared with the prior art. The diversity of the generated scene is improved by 45.39% compared with the prior art. Figure 7 The qualitative comparison between the two-dimensional plane layout of the three-dimensional scene generated by this embodiment and the two-dimensional plane layout generated by the prior art is shown in the attached Figure 7 It can be seen that the buildings in the two-dimensional plane layout generated by this embodiment have clearer boundaries. Figure 8 The following is a qualitative comparison of the image generated by the 3D scene generated in this embodiment and the image rendered in the CARLA simulation software with the image generated by the prior art. Figure 8 It can be seen that the results generated by this embodiment have high visual quality and strong diversity, and there is no problem of inconsistent viewing angles.

[0109] The above-mentioned embodiments only express several implementation methods of the present invention, and the description is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. For ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A controllable generation method of three-dimensional scenes for autonomous driving simulation, characterized in that: include: 1) Obtaining a road network topology map for generating a three-dimensional scene, and constructing a geometric minimum ring set based on the road network topology map; 2) Divide the scene plane corresponding to the road network topology map using the geometric minimum ring as the boundary, obtain multiple sub-regions and assign attribute features to each sub-region, and then generate a binary image corresponding to each sub-region; obtain the trained convolution encoder and use it to extract the encoding vector of the binary image; 3) Obtain the trained graph dynamic attention decoder and use it to decode the attribute features, encoding vectors, and hidden vectors obtained by sampling the standard normal distribution to obtain the graph structure data of each sub-region; 4) Convert the graph structure data into a two-dimensional layout, and stitch all the two-dimensional layouts to obtain the scene plane layout; 5) Build a scene information table based on the scene plane layout and obtain a 3D model library; traverse the floor plan outlines of each building in the scene information table, and select a candidate set of three-dimensional models with outline matching degrees higher than a threshold from the 3D model library; 6) Optimize the matching based on the positional relationship and feature similarity of the buildings in the scene plane layout, and finally perform a 3D model replacement operation to generate the final 3D scene.

2. The controllable generation method of three-dimensional scenes for autonomous driving simulation according to claim 1, characterized in that: In step 2), the trained convolutional encoder is obtained by training the convolutional encoder-decoder using a self-constructed training set. The convolutional encoder-decoder includes a convolutional encoder and a convolutional decoder. The convolutional encoder consists of 4 convolutional layers, and the convolutional decoder is mirror-symmetric to the convolutional encoder.

3. The controllable generation method of three-dimensional scenes for autonomous driving simulation according to claim 1, characterized in that: In step 2), the construction of the training set includes: Obtain multiple road network topology maps in real scenes and scene plane layout vector maps corresponding to the road network topology maps, divide the scene plane corresponding to each scene plane layout vector map into multiple sub-areas according to the method in step 1), and adjust the scaling scale of the scene plane layout vector map of the sub-area so that each building in the sub-area is aligned one by one with each building in the sub-area in the public data set containing building height information; If there is a building with a missing height value in the scene plane layout vector diagram of the sub-area, the building is found in the public data set, and the heights of all buildings within a radius of 300 meters of the building are found from the public data set. The average height of all buildings within a radius of 300 meters is calculated and the average height is used as the height of the building with missing height values. The scene plane layout vector diagram of the sub-area is updated, and finally the scene plane layout vector diagrams of all the sub-areas of the scene plane and the updated sub-areas constitute a data set; then the binary images corresponding to each sub-area in the data set are generated, and the sizes of the obtained binary images are unified. All the unified binary images constitute a training set for training the convolutional codec; The average height and density of buildings in each sub-area in the dataset will also be calculated based on the public dataset, and then attribute characteristics will be assigned to each sub-area based on the calculated results; the attribute characteristics include the density of the building complex and the height of the building complex in the sub-area. The density of the building complex is divided into high density and low density. When the density of buildings in the sub-area is greater than or equal to 0.1, the density of the building complex is high density, otherwise it is low density; the height of the building complex is divided into high-rise and low-rise. When the average height of the building complex is greater than or equal to 20m, it is high-rise, otherwise it is low-rise.

4. The controllable generation method of three-dimensional scenes for autonomous driving simulation according to claim 3, characterized in that: In step 3), obtaining the trained graph dynamic attention decoder includes: Introduce a learnable dynamic attention mechanism into the graph neural network to form a graph dynamic attention encoder, and then construct a graph dynamic attention decoder that is a mirror image of the graph dynamic attention encoder to form a graph dynamic attention encoder-decoder; Normalizing the scene plane layout vector diagram of each sub-region in the data set to obtain graph structure data corresponding to each sub-region; Construct a virtual node connected to all nodes in the graph structure data, obtain the first graph structure data, input the first graph structure data into the graph dynamic attention encoder, and obtain the aggregation vector; obtain the encoding vector and attribute features of the sub-region in the data set, input the encoding vector and attribute features of the sub-region into the graph dynamic attention decoder, and sample the aggregation vector to obtain the hidden vector of the graph dynamic attention decoder, and input the hidden vector into the graph dynamic attention decoder, and finally the graph dynamic attention decoder outputs the second graph structure data; The graph dynamic attention encoder-decoder is trained using conditional variational learning to obtain a trained graph dynamic attention decoder.

5. The controllable generation method of three-dimensional scenes for autonomous driving simulation according to claim 4, characterized in that: The step of inputting the first graph structure data into the graph dynamic attention encoder to obtain an aggregate vector comprises: Each node in the first graph structure data The node characteristics are , ,in, Represents a node Whether there is actually a building unit at the location; Represents a node The type of building units located at the site; Represents a node The horizontal coordinate of the center position of the building unit at; Represents a node The vertical coordinate of the center position of the building unit; Represents a node The height of the building unit at the location; Represents a node The length of the minimum rotation bounding box of the floor plan outline of the building unit at the location; Represents a node The width of the minimum rotation bounding box of the floor plan outline of the building unit at the location; Represents a node The proportion of the floor plan outline of the building unit in the minimum rotation bounding box; First, the node The node characteristics of and neighbor nodes The node characteristics of Concatenate and then use a learnable weight matrix Perform a linear transformation on the concatenated result, use the LeakyReLU activation function on the linear transformation result, and then add it to the attention parameter vector Multiply to get the correlation coefficient ; The specific expression is: ； Then the correlation coefficient The node features of each node are weighted and summed with those of its neighbor nodes, and then passed through a nonlinear activation function to obtain The eigenvector of ; the specific expression is: ； ； in, Represents a node and neighbor nodes The weight coefficient between ; For Node The set of neighbor nodes of ; For Node The The feature vector calculated by the attention layer; is a nonlinear activation function; is the number of attention groups; For the The first attention layer Nodes calculated by the group attention mechanism and neighbor nodes The weight coefficient between It is The first attention layer The input linear transformation matrix of the group attention mechanism; For Node Neighbor nodes The The feature vector calculated by the attention layer, where ; Pass each node through The feature vectors after the attention layer are integrated into the feature matrix , the feature matrices of all attention layers are aggregated into an aggregate vector , .

6. The controllable generation method of three-dimensional scenes for autonomous driving simulation according to claim 5, characterized in that: The loss function when training the graph dynamic attention encoder-decoder using conditional variational learning for: ； in, is the loss function; is a hyperparameter; For the encoding vector , attribute characteristics And the first graph structure data Under the condition of The expectation of the posterior distribution of ; Represented in the encoding vector , attribute characteristics And the first graph structure data Under the condition of The posterior distribution of For the encoding vector , attribute characteristics and the hidden vector Under the condition of Distribution of is the hidden vector The prior distribution of express and The KL divergence distance between two distributions; is a hyperparameter; Represents the attribute att of the building in the first graph structure data of the input graph dynamic attention encoder; Represents the attribute att of the building in the second graph structure data output by the graph dynamic attention decoder; is the L2 norm.

7. The controllable generation method of three-dimensional scenes for autonomous driving simulation according to claim 1, characterized in that: In step 5), the scene information table includes the ID identification of the building unit in the sub-area, the sub-area ID, the type of the building unit, the plan outline of the building unit, the height and area of ​​the building unit; The 3D building library includes 3D models of various shapes and materials, and each 3D model is tilted from the front view. The images are taken from eight viewing angles and a top-down outline of each 3D model.

8. The controllable generation method of three-dimensional scenes for autonomous driving simulation according to claim 7, characterized in that: In step 5), the step of selecting three-dimensional models with contour matching degrees higher than a threshold from the 3D model library to construct a candidate set includes: First, the floor plan outlines of each building in the scene information table and the top view outlines of the three-dimensional model in the 3D model library are traversed to calculate the matching degree between the floor plan outline of the building and the top view outline of the three-dimensional model. The matching degree calculation formula is: ； in, The top view of the 3D model The length of the minimum outer bounding box and width In the case of a building with a floor plan outline Top view of the 3D model The degree of matching; The floor plan outline of the building Top view of the 3D model The intersection and union ratio of The top view of the 3D model The length of the minimum outer bounding box and width In the case of a building with a floor plan outline Top view of the 3D model The area of ​​the intersection; Represents the floor plan outline of a building The area of Represents the top view of the 3D model The length of the minimum outer bounding box and width In the case of area; The maximum intersection-union ratio is used as the cost function, and the Powell method is used to minimize the cost function through iterative search to obtain the top view outline of the 3D model with the best matching degree. The length and width of the minimum outer bounding box of the 3D model are calculated; finally, the intersection and union ratio between the top-view outline of the 3D model and the floor plan outline of the building under the best matching degree is calculated, and the 3D model with an intersection and union ratio greater than the set threshold is taken as a candidate in the candidate set of the building.

9. The controllable generation method of three-dimensional scenes for autonomous driving simulation according to claim 8, characterized in that: In step 6), the optimization matching based on the position relationship and feature similarity of the buildings in the scene plane layout includes: The CLIP image encoder is used to encode the pictures of each view of the 3D model in the 3D model library to obtain feature vectors, and then the feature vectors of each view are spliced ​​to obtain the encoding vector of the 3D model, that is, the material characteristics of the 3D model are obtained; Obtain the material features of all 3D models in the 3D model library, then extract the material features of each 3D model in the candidate set to form a material feature set corresponding to the corresponding candidate set; construct a cost function with the goal that the material features of buildings in the same sub-area of ​​the scene layout are similar, while the material features of buildings in different sub-areas are different Optimize matching; cost function The formula is: ； ； ； ； in, is the kth building in the scene plane; is the number of 3D models in the candidate set of the kth building; To indicate whether the i-th 3D model in the candidate set of the k-th building is selected; Represents the material features of the i-th 3D model in the candidate set of the k-th building; Indicates a building and Are they in the same sub-region? -1 means in the same sub-area. A value of 1 indicates that they are not in the same sub-area.

10. A controllable three-dimensional scene generation system for autonomous driving simulation that implements the method of claim 1, characterized in that: include: A coding vector and attribute feature acquisition module, which is used to obtain a road network topology map to be used for three-dimensional scene generation, and to construct a geometric minimum ring set based on the road network topology map; Divide the scene plane corresponding to the road network topology map using the geometric minimum ring as the boundary, obtain multiple sub-areas and assign attribute features to each sub-area, and then generate a binary image corresponding to each sub-area; obtain the trained convolution encoder and use it to extract the encoding vector of the binary image; A graph structure data acquisition module is used to obtain a trained graph dynamic attention decoder and use it to decode attribute features, encoding vectors, and hidden vectors obtained by sampling the standard normal distribution to obtain graph structure data for each sub-region; A scene plane layout acquisition module, which is used to convert the graph structure data into a two-dimensional layout, and to splice all the two-dimensional layouts to obtain the scene plane layout; A candidate set acquisition module is used to construct a scene information table based on the scene plane layout and obtain a 3D model library; traverse the floor plan outlines of each building in the scene information table, and select three-dimensional models with outline matching degrees higher than a threshold from the 3D model library to construct a candidate set; The 3D scene generation module is used to optimize the matching based on the position relationship and feature similarity of the buildings in the scene plane layout, and finally perform the 3D model replacement operation to generate the final 3D scene.

Images