A three-dimensional semantic map construction method for a city information model

Abstract

The application relates to a three-dimensional semantic map construction method for a city information model, which comprises the following steps: S1, designing and training a deep neural network while extracting semantic segmentation information and SLAM features of an input image; S2, inputting the semantic segmentation information and the SLAM features into an improved binocular ORB-SLAM3 framework to estimate camera pose and continuously incrementally update a sparse point cloud semantic map; S3, performing space-semantic map point clustering on the sparse point cloud semantic map generated in S2 in combination with the semantic segmentation information; S4, reconstructing an object model in the form of a hyperquadric object according to the space-semantic map point clustering result and continuously incrementally optimizing the object model; S5, constructing an object-level three-dimensional semantic map based on the reconstructed hyperquadric object model and the camera pose estimation; and S6, obtaining an RGB-D image based on binocular stereo matching, constructing a dense point cloud-level three-dimensional semantic map in combination with the camera pose estimation and the semantic segmentation information. Compared with the prior art, the method has higher precision and robustness and stronger environment representation capability.

Description

Technical Field

[0001] This invention relates to the fields of digital twin and SLAM technology, and in particular to a method for constructing three-dimensional semantic maps for urban information models. Background Technology

[0002] Digital twins refer to the technology of fully utilizing physical models, sensors, operational history, and other data to integrate multi-disciplinary, multi-physical, multi-scale, and multi-probabilistic simulation processes, mirroring physical entities in a virtual information space to reflect the entire lifecycle of the physical entity's behavior, state, or activities. The more complex the system, the greater the improvement in management efficiency and the greater the benefits after establishing a digital twin. Cities are the most complex and massive systems, and digital twin cities have emerged as a result. The "14th Five-Year Plan for National Economic and Social Development and the Long-Range Objectives Through the Year 2035" explicitly proposes to "explore the construction of digital twin cities" and to "use digitalization to promote urban and rural development and governance model innovation, and comprehensively improve operational efficiency and livability."

[0003] City Information Modeling (CIM) is a crucial component of the infrastructure side of digital twin city technology. It includes virtual models and attribute information of natural landforms such as mountains and rivers, infrastructure such as roads and bridges, and various buildings. As cities develop rapidly, CIMs must also be continuously updated. Using data collection vehicles equipped with multiple sensors to traverse the urban environment and construct 3D semantic maps is an important means of building CIMs.

[0004] SLAM (Simultaneous Localization and Mapping) is the fundamental method for vehicles to autonomously build 3D urban maps. Binocular cameras can acquire information-rich color images, and depth maps can be reconstructed through stereo matching. Their cost is far lower than LiDAR, making them suitable as the primary sensor for city-level SLAM. ORB-SLAM3 is currently the best-performing open-source visual SLAM solution and supports binocular image input; therefore, this invention chooses it as the basic framework for the algorithm. However, ORB-SLAM3 can only build sparse point cloud-level geometric maps; further improvements are needed to construct semantic maps.

[0005] Currently, there has been some research on building semantic maps using SLAM, but some problems still exist.

[0006] Semantic maps can be categorized into point cloud level and object level. Point cloud level semantic maps emerged earlier, requiring only a category label for each map point, making their construction relatively simple. However, such maps contain too low a level of semantic information to generate practical urban information models. Object level semantic maps, on the other hand, include objects in the environment, along with their categories, outlines, poses, and other parameters, more closely resembling human environmental perception. They can effectively support the generation of urban information models; however, object-level maps are not suitable for representing uncountable background elements such as roads.

[0007] The construction of a semantic map first requires a semantic extractor. Existing object-level semantic SLAM generally uses object detection networks such as the YOLO series as semantic extractors, which can provide several rectangular object detection boxes. From these detection boxes, parameters such as object size and pose can be recovered in a relatively simple way. However, rectangular detection boxes cannot represent the accurate boundaries of objects, making it difficult to recover the shape parameters of objects.

[0008] Existing object-level semantic SLAM primarily employs cubes and quadric surfaces as object models. These are two relatively regular object models with fewer parameters, and when combined with the bounding boxes provided by the aforementioned object detection networks, the computational efficiency for parameter generation and optimization is high. However, objects in real-world urban environments generally have complex shapes, and the aforementioned models struggle to effectively represent irregular objects, thus affecting the versatility of semantic map construction methods.

[0009] Currently, there is an urgent need to study a semantic SLAM algorithm that can fully represent as many different types of environmental objects as possible, build multi-level city maps, and balance accuracy, robustness, and real-time performance. Summary of the Invention

[0010] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a method for constructing a three-dimensional semantic map for urban information models with higher accuracy and robustness and stronger environmental representation capabilities. This method uses hyperquadratic surfaces as environmental object models and constructs a multi-level semantic map, including object-level maps, from semantic segmentation.

[0011] The objective of this invention can be achieved through the following technical solutions:

[0012] This invention provides a method for constructing a three-dimensional semantic map based on a city information model, characterized by the following steps:

[0013] Step S1: Design and train a deep neural network to extract semantic segmentation information and SLAM features from the input image.

[0014] Step S2: Input the semantic segmentation information and SLAM features obtained in step S1 into the improved binocular ORB-SLAM3 framework to perform camera pose estimation and continuously incrementally update the sparse point cloud semantic map.

[0015] Step S3: Perform spatial-semantic map point clustering on the sparse point cloud semantic map generated in step S2, combined with the semantic segmentation information in step S1.

[0016] Step S4: Based on the spatial-semantic map point clustering results, reconstruct the object model in the form of a hyperquadratic surface, and continuously incrementally optimize the object model;

[0017] Step S5: Based on the reconstructed hyperquadratic surface object model and camera pose estimation, construct an object-level 3D semantic map;

[0018] Step S6: Based on binocular stereo matching, acquire RGB-D images, and combine camera pose estimation and semantic segmentation information to construct a dense point cloud-level 3D semantic map.

[0019] Preferably, the construction process of the sparse point cloud semantic map, the object-level 3D semantic map, and the dense point cloud-level 3D semantic map is a three-parallel process.

[0020] Preferably, the architecture of the deep neural network in step S1 is as follows: a network branch is added to the end of the HRNetV2 network used for semantic segmentation, which outputs the key points and descriptors required for localization and mapping, forming a multi-branch network, namely the semantic segmentation branch and the SLAM feature extraction branch.

[0021] The semantic segmentation branch is trained using the cross-entropy loss function; the SLAM feature branch is trained using a keypoint ground truth construction method and loss function based on Superpoint.

[0022] Preferably, the improved binocular ORB-SLAM3 framework in step S2 has the following structure: based on the ORB-SLAM3 framework structure, the feature extractor is replaced with the deep neural network in step S1.

[0023] Preferably, the distance metric used in the spatial-semantic map point clustering in step S3 consists of the spatial distance between map points and the probability distribution difference of semantic attributes between map points, specifically expressed as:

[0024]

[0025] In the formula, It is a sparse map of points constructed from ORB-SLAM3 model. Map point Three-dimensional coordinates in the world coordinate system Representing map points The probability distribution of semantic category labels, Represents the Kullback-Leibler divergence. For weight hyperparameters.

[0026] Preferably, the object model in the form of a hyperquadratic surface in step S4 is specifically expressed as follows:

[0027] The vector representation of a hyperquadratic surface is as follows:

[0028]

[0029] In the formula, These are the size parameters in three dimensions. For shape parameters; , For iteration variables;

[0030] The constraint expression for the hyperquadratic surface is:

[0031]

[0032] In the formula, For dimensional parameters, For shape parameters;

[0033] The object parameters of the hyperquadratic surface object model also include pose parameters. .

[0034] Preferably, step S4 includes the following sub-steps:

[0035] Step S41: Perform principal component analysis (PCA) on the clustering results of the initial map points to initialize the object parameters in the hyperquadratic surface form. ;in, These are the size parameters in three dimensions. For shape parameters; The object's pose;

[0036] Step S42: Optimize object parameters with the goal of minimizing the projection of the corresponding object in the map onto the current frame's object image region.

[0037] Step S43: Optimize object parameters with the goal of minimizing the distance between the object map point and the object surface in the current frame.

[0038] Preferably, the optimization process in steps S41 and S42 adopts a phased optimization method, specifically: first, fix the size parameters and optimize other parameters, then fix the other parameters and optimize the size parameters, and finally jointly optimize all object parameters.

[0039] Preferably, step S5 further includes: training corresponding dictionaries for all possible object categories output by semantic segmentation, and completing object data association based on the bag-of-words model, including the following sub-steps:

[0040] 1) Identify candidate related objects among objects in a known map based on the cluster center location;

[0041] 2) Using the bag-of-words matching method, based on the dictionary of the object categories to which the map point cluster belongs, the association between the map point cluster in the current frame and the objects in the known map is completed;

[0042] 3) After reconstructing the object from the map point cluster, complete the object data association.

[0043] Preferably, step S6 specifically comprises:

[0044] 1) For each keyframe, stereo matching is performed using the LEAStereo algorithm to obtain the depth map corresponding to the RGB image;

[0045] 2) Combine the semantic segmentation label map to obtain the dense semantic point cloud corresponding to the keyframe; the category labels of map points are updated using a recursive Bayesian algorithm;

[0046] 3) The dense semantic point clouds of each keyframe are stitched together according to the camera pose estimation to obtain a dense point cloud-level semantic map; wherein, the camera pose of each keyframe is calculated by the ORB-SLAM3 model.

[0047] Compared with the prior art, the present invention has the following advantages:

[0048] 1) This invention uses hyperquadratic surfaces as object models in semantic maps and designs corresponding generation and optimization algorithms, which can effectively represent more objects with complex sizes and shapes, improve the versatility of semantic maps, and make digital twin city information models closer to complex real urban environments.

[0049] 2) This invention reconstructs objects from semantic segmentation, rather than the target detection commonly used in previous solutions, which can obtain more accurate object contours, improve the accuracy of semantic maps, and make the digital twin city information model closer to the complex real urban environment.

[0050] 3) This invention uses the same deep neural network to simultaneously output SLAM features and semantic segmentation results. Compared with previous semantic SLAM schemes, the algorithm is simpler and has higher running efficiency.

[0051] 4) This invention uses an improved bag-of-words model to associate object data, achieving a good balance between accuracy and efficiency, enabling semantic maps to maintain accuracy and robustness in large-scale environments, thus allowing digital twin city information models to be used in cities with larger areas.

[0052] 5) This invention establishes a multi-layered geometric-semantic map to meet the needs of urban information models with different requirements. Attached Figure Description

[0053] Figure 1 This is a system framework diagram of the present invention;

[0054] Figure 2 This is a diagram of the deep neural network structure used for semantic segmentation and SLAM feature extraction in this invention;

[0055] Figure 3 This is a schematic diagram of the hyperquadratic surface used as an object model in this invention. Detailed Implementation

[0056] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0057] Example

[0058] like Figure 1 As shown, this embodiment provides a method for constructing a 3D semantic map based on a city information model. The method specifically includes the following steps:

[0059] Step S1: Train a high-performance deep neural network to simultaneously extract semantic segmentation information and features required for localization and sparse geometric mapping from the input image;

[0060] like Figure 2 As shown, the deep neural network used in this invention is developed based on the HRNetV2 network. HRNetV2 is a high-performance semantic segmentation network. Originally, the low-resolution output of the four different resolution outputs of the network was bilinearly upsampled to the size of the high-resolution output (i.e., the input image resolution), and these four high-resolution outputs were concatenated by a 1x1 convolution. After one convolution, it was used as the category probability output for semantic segmentation.

[0061] This invention inputs the combined high-resolution output into three convolutional layers respectively, and outputs semantic segmentation results (including a single-layer semantic segmentation label map and a multi-layer category probability distribution map), key points and corresponding descriptors.

[0062] The training set is the Cityscapes dataset, which contains ground truth pixel semantic labels. Similar to Superpoint, this invention employs the MagicPoint keypoint detector and performs self-annotation of ground truth keypoints on the dataset; simultaneously, it uses adaptive homography transformation to augment the dataset, where the transformation intensity parameter is set to... Correspondingly, the same homography transformation should also be performed on the semantic segmentation truth graph.

[0063] Therefore, the final dataset used includes: original Cityscapes images, images after random homography transformation, ground truth semantic segmentation values, and ground truth keypoint values. The semantic segmentation branch of the network is trained using the standard cross-entropy loss function; the SLAM feature extraction branch of the network is trained using the same loss function as Superpoint.

[0064] The output semantic segmentation results include: a 1-layer semantic segmentation label map and a 19-layer semantic segmentation category probability distribution map. The output SLAM features include keypoints and corresponding 128-dimensional descriptors.

[0065] Step S2: Input the semantic segmentation information and SLAM features obtained in Step S1 into the improved binocular ORB-SLAM3 framework to perform camera pose estimation and continuously incrementally update the sparse point cloud semantic map. The specific process is as follows:

[0066] The architecture of the improved binocular ORB-SLAM3 model of the present invention is as follows: based on the ORB-SLAM3 model structure, the feature extractor is replaced with the deep neural network in step S1.

[0067] The SLAM framework is developed based on ORB-SLAM3. It takes stereo images and camera parameters (including camera intrinsic matrix and stereo camera baseline) as input, inputs the left eye image into a deep neural network, and outputs semantic segmentation results and features required for SLAM. These features replace ORB features input into ORB-SLAM3 and output the camera pose of each frame (6-DoF pose trajectory of the left eye camera). It continuously and incrementally updates the sparse geometric point cloud map.

[0068] Step S3: Perform spatial-semantic map point clustering on the sparse point cloud semantic map generated in step S2, combined with the semantic segmentation information in step S1.

[0069] The map points output by the modified binocular ORB-SLAM3 are given 3D coordinates and semantic labels. Based on these two attributes, the map points are clustered, and map points that may belong to the same object are grouped into clusters.

[0070] For any two map points The distance metric required for clustering is:

[0071]

[0072] in, For weight hyperparameters, Map point Three-dimensional coordinates in the world coordinate system It measures the Euclidean distance between two map points in space. Representing map points The probability distribution of semantic category labels, Represents the Kullback-Leibler divergence. This can be referred to as the "semantic distance" between two map points, and the specific calculation method is as follows:

[0073]

[0074] Furthermore, since there is a correspondence between sparse 3D map points and image feature points on keyframes, semantic-spatial clustering of map points also means clustering feature points on the image. Then, by performing connected component analysis on the corresponding semantic segmentation result graph, several semantic masks of different objects in the corresponding space can be obtained on the image.

[0075] The construction of the object-level semantic map is a new thread. Each keyframe generates sparse geometric map points that correspond to feature points in the image, thus assigning corresponding semantic category labels to the map points. Connectivity analysis is performed on the semantic segmentation result map of the frame using OpenCV, discarding uncountable or difficult-to-represent object categories such as sky and road surfaces, and discarding connected components with an area less than 0.05% of the image size. The remaining number of connected components is the preset number of clusters, and each connected component represents the semantic mask corresponding to the object. Then, spatial-semantic clustering is performed on the map points, with OPTICS chosen as the clustering method, and the hyperparameter in the distance metric set to [value missing]. Clustering yields map point clusters corresponding to a series of different objects.

[0076] Step S4: Reconstruct the object model in the form of a hyperquadratic surface from the spatial-semantic map point clustering results;

[0077] For each map point cluster generated in step S3, based on the camera pose of that frame, the object parameters of the hyperquadratic surface model are initialized using a PCA-like method. As the map point cluster is incrementally updated and a series of observed images of the object and their camera poses are recorded, the object parameters are continuously optimized. An example of a hyperquadratic surface is shown below. Figure 3 .

[0078] Hyperquadrilaterals are an extension of standard quadric surfaces, adding shape parameters to represent objects with more diverse shapes. A point on the surface of a hyperquadrilateral... The basic parameter expression is:

[0079]

[0080] in, These are the size parameters in three dimensions. For shape parameters, , For iteration variables.

[0081] The constraint expression for the hyperquadratic surface is:

[0082]

[0083] Any point in the object's coordinate system Substituting into the above formula, if Then the point lies on the surface of the hyperquadratic surface; if Then the point lies outside the hyperquadratic surface; if If the point is located inside the hyperquadratic surface, then the point is located inside the hyperquadratic surface.

[0084] Hyperquadratic surfaces in three-dimensional space and pose parameters in the world coordinate system It also defines a transformation from the world coordinate system to the object coordinate system. Therefore, an object in space represented by a hyperquadratic surface has the following parameters: .

[0085] The specific process for calculating and optimizing the object model parameters is as follows:

[0086] Step S41: Perform principal component analysis on the clustering results of the initial map points to initialize the parameters of the object model. ;

[0087] For each map point cluster obtained by clustering After point cloud decentralization, it is denoted as a matrix. Then, the initial values ​​of the hyperquadratic surface parameters are obtained as follows:

[0088]

[0089]

[0090]

[0091]

[0092] In addition, for object pose Initialization, position The initial value is the position of the center point of the semantic mask region on the image corresponding to the object in the world coordinate system, given by ORB-SLAM3; pose Pick The result is converted to Euler angles. This completes the initialization of the object parameters.

[0093] Step S42: Optimize object parameters with the goal of minimizing the projection of the corresponding object in the map onto the current frame's object image region.

[0094] Object parameters The optimization is defined as:

[0095]

[0096] in, For image sequences The corresponding camera pose, For image Upper corresponding object semantic mask region, function It refers to objects in space Reprojection onto camera pose is Image Above, and Calculate the reprojection error; for The constraints are to ensure that the object's shape is convex and its size is not too small, thus ensuring the optimization effect.

[0097] Step S43: Optimize object parameters with the goal of minimizing the distance between the object map point and the object surface in the current frame;

[0098] Object parameters The optimization is defined as:

[0099]

[0100] in, For corresponding objects The map point clusters are then transformed to the object coordinate system. For map points To the object Surface distance measurement, coefficient This is to select the group with the smallest size among the candidate object parameters.

[0101] In step S4, the optimization of object parameters is carried out in a multi-stage manner. First, the shape parameters are fixed. Optimize other parameters; then fix the other parameters and optimize the shape parameters. Finally, all object parameters are optimized simultaneously. The optimization process is as follows: First, using the method in step S42, optimization is performed considering all keyframes with known poses, where the function... The calculation method is as follows: 100 points are randomly sampled from the hyperquadratic surface and projected onto the image. The boundary is calculated using OpenCV, and then the intersection-union ratio (IUU) is calculated with the semantic mask of the object to obtain the object reprojection error. Then, in step S43, all map points corresponding to the object are considered for optimization, and the hyperparameter values ​​are determined. .

[0102] Step S5: Combine object reconstruction and camera pose to construct an object-level 3D semantic map;

[0103] In steps S4 and S5, before actual execution, the deep neural network from S1 is used to perform semantic segmentation and feature extraction on the KITTI visual odometry dataset. Then, a DBoW library, similar to ORB-SLAM3, is used to calculate a feature dictionary for each possible category output by the semantic segmentation. During algorithm execution, these dictionaries are used to perform object data association. For each map point cluster in the current frame, all objects that can contain the cluster center are considered candidate associated objects. A bag-of-words model is then used to filter out the final associated objects from the candidate associated objects, updating the map points corresponding to those objects and optimizing the object parameters.

[0104] Step S6: Generate an RGB-D image from the input binocular image, and construct a point cloud-level 3D semantic map by combining camera pose and semantic segmentation;

[0105] The construction of the dense point cloud-level semantic map is a new thread. For each keyframe, stereo matching is performed using the LEAStereo algorithm to obtain the depth map corresponding to the RGB image. This depth map is then combined with the semantic segmentation label map to obtain the dense semantic point cloud for that frame. The camera pose for each keyframe is calculated using ORB-SLAM3. The dense semantic point clouds of each frame are stitched together according to their poses to obtain the dense point cloud-level semantic map. The category labels of the map points are updated using a recursive Bayesian approach.

[0106] Map Points Category tags for:

[0107]

[0108] in, Map points in the current frame The probability distribution of the category labels, It is the map point up to the previous frame. The probability distribution of the category labels, These are the normalized parameters.

[0109] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for constructing a three-dimensional semantic map for urban information models, characterized in that, The method includes the following steps: Step S1: Design and train a deep neural network to extract semantic segmentation information and SLAM features from the input image. Step S2: Input the semantic segmentation information and SLAM features obtained in step S1 into the improved binocular ORB-SLAM3 framework to perform camera pose estimation and continuously incrementally update the sparse point cloud semantic map. Step S3: Perform spatial-semantic map point clustering on the sparse point cloud semantic map generated in step S2, combined with the semantic segmentation information in step S1. Step S4: Based on the spatial-semantic map point clustering results, reconstruct the object model in the form of a hyperquadratic surface, and continuously incrementally optimize the object model; Step S5: Based on the reconstructed hyperquadratic surface object model and camera pose estimation, construct an object-level 3D semantic map; Step S6: Based on binocular stereo matching, acquire RGB-D images, and combine camera pose estimation and semantic segmentation information to construct a dense point cloud-level 3D semantic map.

2. The method for constructing a three-dimensional semantic map for urban information models according to claim 1, characterized in that, The construction process of the sparse point cloud semantic map, the object-level 3D semantic map, and the dense point cloud-level 3D semantic map consists of three parallel processes.

3. The method for constructing a three-dimensional semantic map for urban information models according to claim 1, characterized in that, The architecture of the deep neural network in step S1 is as follows: a network branch is added to the end of the HRNetV2 network used for semantic segmentation, which outputs the key points and descriptors required for localization and mapping, forming a multi-branch network, namely the semantic segmentation branch and the SLAM feature extraction branch. The semantic segmentation branch is trained using the cross-entropy loss function; the SLAM feature branch is trained using a keypoint ground truth construction method and loss function based on Superpoint.

4. The method for constructing a three-dimensional semantic map for urban information models according to claim 1, characterized in that, In step S2, the binocular ORB-SLAM3 framework is improved. Specifically, the feature extractor in the ORB-SLAM3 framework is replaced with the deep neural network in step S1.

5. The method for constructing a three-dimensional semantic map for urban information models according to claim 1, characterized in that, In step S3, the distance metric used for spatial-semantic map point clustering consists of the spatial distance between map points and the probability distribution difference of semantic attributes between map points, specifically expressed as: In the formula, It is a sparse map of points constructed from ORB-SLAM3 model. Map point Three-dimensional coordinates in the world coordinate system Representing map points The probability distribution of semantic category labels, Represents the Kullback-Leibler divergence. For weight hyperparameters.

6. The method for constructing a three-dimensional semantic map for urban information models according to claim 1, characterized in that, The specific expression for the hyperquadratic surface-form object model in step S4 is as follows: The vector representation of a hyperquadratic surface is as follows: In the formula, These are the size parameters in three dimensions. For shape parameters; , For iteration variables; The constraint expression for the hyperquadratic surface is: In the formula, For dimensional parameters, For shape parameters; The object parameters of the hyperquadratic surface object model also include pose parameters. .

7. The method for constructing a three-dimensional semantic map for urban information models according to claim 1, characterized in that, Step S4 includes the following sub-steps: Step S41: Perform principal component analysis (PCA) on the clustering results of the initial map points to initialize the object parameters in the hyperquadratic surface form. ;in, These are the size parameters in three dimensions. For shape parameters; The object's pose; Step S42: Optimize object parameters with the goal of minimizing the projection of the corresponding object in the map onto the current frame's object image region. Step S43: Optimize object parameters with the goal of minimizing the distance between the object map point and the object surface in the current frame.

8. A method for constructing a three-dimensional semantic map for urban information models according to claim 7, characterized in that, The optimization processes in steps S41 and S42 both adopt a phased optimization approach, specifically: first, fix the size parameters and optimize other parameters; then, fix the other parameters and optimize the size parameters; finally, jointly optimize all object parameters.

9. A method for constructing a three-dimensional semantic map for urban information models according to claim 1, characterized in that, Step S5 further includes: training corresponding dictionaries for all possible object categories in the semantic segmentation output, and completing object data association based on the bag-of-words model, including the following sub-steps: 1) Identify candidate related objects among objects in a known map based on the cluster center location; 2) Using the bag-of-words matching method, based on the dictionary of the object categories to which the map point clusters belong, the association between the map point clusters of the current frame and the objects in the known map is completed; 3) After reconstructing the object from the map point cluster, complete the object data association.

10. A method for constructing a three-dimensional semantic map for urban information models according to claim 1, characterized in that, Step S6 specifically involves: 1) For each keyframe, stereo matching is performed using the LEAStereo algorithm to obtain the depth map corresponding to the RGB image; 2) Combine the semantic segmentation label map to obtain the dense semantic point cloud corresponding to the keyframe; the category labels of map points are updated using a recursive Bayesian algorithm; 3) The dense semantic point clouds of each keyframe are stitched together according to the camera pose estimation to obtain a dense point cloud-level semantic map; wherein, the camera pose of each keyframe is calculated by the ORB-SLAM3 model.

Images