Method for reconstructing three-dimensional building by using radar point cloud

Through the timestamp matching and reflectivity screening of point clouds and images, the building facade is refined in combination with the density clustering algorithm to generate a three-dimensional architectural model with real texture, which solves the accuracy and texture of point cloud three-dimensional reconstruction in the existing technology Simple questions improve the realistic and processing efficiency of the model.

CN118918252AActive Publication Date: 2024-11-08ZHEJIANG UNIV
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
CN202410950028.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-16
Publication Date
2024-11-08
Estimated Expiration
2044-07-16

AI Technical Summary

Technical Problem

The existing technology has huge data scale, noise interference and complex structure resolution in point cloud three-dimensional reconstruction, resulting in insufficient precision in reconstruction results and simple surface texture, which cannot meet the needs of high-quality building information models.

Method used

Time stamp matching is performed through the offset effect of point clouds and images, combined with reflectivity threshold filtering and noise-resistant density clustering algorithm, the semantic features of the building are extracted and the facade structure is refined, and a three-dimensional building with real texture is generated using the Blender tool. Model.

Benefits of technology

It realizes high-precision texture mapping, improves the realism and detail restoration capabilities of the model, improves point cloud processing efficiency and visual effects of the model, simplifies the texture addition process, and reduces manual intervention and error rates.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918252A_ABST
    Figure CN118918252A_ABST
Patent Text Reader

Abstract

The invention discloses a method for reconstructing a three-dimensional building by using radar point cloud. The method comprises the following steps of: firstly, aligning timestamps of a point cloud and an image by utilizing offset of the point cloud and the image in vision to obtain a mapping relation between the point cloud and the image; thirdly, splicing and cutting the point clouds with continuous time tracks according to poses to obtain final building point clouds, filtering the point clouds by utilizing a reflectivity threshold value, extracting building semantic features, and distinguishing the point clouds through various building prior knowledge to generate structured facade information representation; and finally, converting the facade information into a building information model, projecting texture features, and finally forming a three-dimensional building model with textures. According to the method, the application of directly generating the three-dimensional building model from the point cloud is realized, the real-time processing capability and the innovative processing strategy are integrated in operation, the method is widely applied to multiple fields such as building design, urban planning and cultural heritage protection, and an efficient and accurate new way is provided for three-dimensional building reconstruction.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a point cloud three-dimensional reconstruction method in the technical field of three-dimensional modeling, and in particular to a method for rapidly reconstructing a three-dimensional building using a radar point cloud. Background Art

[0002] In the field of modern three-dimensional spatial data processing, point clouds, as the core carrier, provide detailed geometric and topological information for the accurate modeling of complex buildings. The main obstacles faced by this technology involve the large scale of data, noise interference, and the difficulty of analyzing complex structures, which drives researchers to innovate and develop efficient and robust algorithms to optimize the point cloud processing process and ensure the accuracy of three-dimensional reconstruction. For example, traditional methods based on geometric features have gradually transitioned to the use of deep learning technology, especially graph convolutional networks (GCNs), which effectively capture the local structural features of point clouds, especially on large-scale data sets.

[0003] In terms of point cloud acquisition technology, Structure from Motion (SfM) and Multi-View Stereo (MVS) have become mainstream. The former uses multi-image feature matching to reconstruct sparse point clouds, and the latter generates high-density point clouds through multi-view image stereo matching, which together support the basic framework of the building's three-dimensional model. On this basis, Poisson surface reconstruction, combined with advanced technologies such as texture mapping, can directly construct highly detailed 3D mesh models from point clouds. Although these models still have defects, with the advancement of semantic understanding and post-optimization technology, the integrity of the model is being significantly improved.

[0004] After acquiring the point cloud, most of the work is to perform semantic segmentation on the point cloud and then reconstruct it. However, the reconstruction result is often composed of multiple cubes stacked together, and the surface texture is only achieved through simple projection, which is still a gap from high-quality building information models.

[0005] Digital building models show great potential in many cutting-edge application scenarios, such as digital urban management, game development, navigation map construction, and autonomous driving simulation training. In order to enhance the realism and practicality of the model, research is focusing on the deep fusion of images and point clouds. For example, using deep learning-driven image feature extraction and texture mapping strategies to directly integrate high-resolution textures into digital building models, greatly enhancing the realism and market application value of the model. In addition, the integration of real-time rendering technology and ray tracing algorithms is also further pushing the boundaries of model interactivity and visual effects, bringing users an immersive experience. Summary of the invention

[0006] In order to solve the problems existing in the background technology, the present invention provides a method for reconstructing three-dimensional buildings using radar point clouds, which is used to construct three-dimensional building models. The present invention is applicable to building data with existing laser radar point clouds and images, and is particularly suitable for industry applications such as architectural design, urban planning, and cultural heritage protection.

[0007] The technical solution adopted by the present invention comprises the following steps:

[0008] 1. A method for reconstructing 3D buildings using radar point clouds

[0009] S1: Using the offset effect of point cloud and image, the time stamps of point cloud and image are matched to obtain the mapping relationship between point cloud and image; the texture features of image are also extracted;

[0010] S2: Based on the pose information on the continuous time trajectory, the point cloud is spliced ​​and cropped to extract the complete building point cloud; then the reflectivity threshold screening mechanism is used to filter the complete building point cloud to obtain the processed building point cloud;

[0011] S3: Use a noise-resistant density-based spatial clustering algorithm to perform semantic recognition on the processed building point cloud to obtain various building semantic features; then combine the building structure attributes and the recognized building semantic features to refine the facade structure of the processed building point cloud to obtain complete building facade information;

[0012] S4: The complete building facade information is converted into a building information model, and then texture features are added to the building information model according to the mapping relationship between the point cloud and the image, thereby generating a three-dimensional building model with real texture.

[0013] In S1, feature points in the image and feature points in the point cloud are identified, the feature points in the image are matched with corresponding features in the point cloud, and then the mapping relationship between the point cloud and the image is obtained after minimizing the spatiotemporal deviation between the two through an optimization algorithm.

[0014] In S2, the complete building point cloud is filtered using a reflectivity threshold screening mechanism to obtain a processed building point cloud, including:

[0015] The reflectivity value I in the complete building point cloud p Point clouds p with reflectivity less than the reflectivity threshold T are removed.

[0016] In S3, the facade structure of the processed building point cloud is refined by combining the building structure attributes and the recognized building semantic features to obtain complete building facade information, specifically:

[0017] According to the feature repetition and equal spacing of architectural semantic features in the building structure, the feature missing parts and erroneous features in the facade structure of the processed building point cloud are fine-tuned to obtain complete building facade information.

[0018] In S4, the complete building facade information is converted into a building information model using a Blender modeling tool.

[0019] In S4, according to the mapping relationship between the point cloud and the image, the texture features of the image are fitted to the corresponding surface of the BIM model by means of image projection, and then the realism of the model is enhanced by adjusting the lighting model and the material, thereby generating a three-dimensional building model with real texture.

[0020] 2. A computer device

[0021] The computer device comprises a memory and a processor, wherein the memory stores a computer program, and the steps of the method are implemented when the processor executes the computer program.

[0022] 3. A computer-readable storage medium

[0023] A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method described are implemented.

[0024] IV. A COMPUTER PROGRAM PRODUCT

[0025] The computer program product comprises a computer program / instructions which, when executed by a processor, implement the steps of the method.

[0026] The beneficial effects of the present invention are:

[0027] Precise time-space matching improves texture mapping quality: By applying the visual offset effect of point cloud and image, the present invention achieves high-precision matching of point cloud and image timestamps, and builds a fine correspondence between point cloud and image. This not only ensures the accuracy of model texture mapping, but also greatly enhances the realism and detail restoration capabilities of the reconstructed model, especially in complex lighting and dynamic environments.

[0028] Efficient point cloud processing and structural optimization: Under the guidance of continuous time trajectory, the point cloud is intelligently spliced ​​and cropped, combined with the reflectivity threshold screening mechanism, effectively filtering out noise interference and retaining pure building structure information. This strategy not only improves the quality of the point cloud, but also accelerates the subsequent processing process, making the reconstructed model clearer and more accurate.

[0029] Advanced semantic understanding and refined modeling: Using the advanced noise-resistant density clustering algorithm DBSCAN, it is possible to cluster point clouds that may contain specific semantic features such as doors, windows, and roofs. Based on the point clouds combined with architectural expertise, the building structure is further constructed and the facade details are further refined. This not only deepens the structural understanding level of the model, but also provides a richer information foundation for subsequent building design, analysis, and management.

[0030] BIM model generation with enhanced realistic textures: Using professional modeling tools such as Blender and combining image projection technology, this invention can fit precisely matched textures to the corresponding surfaces of the BIM model. Through fine lighting model and material adjustment, the realism and visual effects of the model are greatly improved. This process not only simplifies the tedious steps of traditional BIM texture addition, but also makes the model more visually expressive and practical.

[0031] Automated processing: The method proposed in the present invention can be converted into an executable software solution through the implementation form of computer equipment and computer-readable storage media, which greatly promotes the automation process of three-dimensional building modeling, reduces the cost and error rate of manual intervention, and improves overall work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is an overall flow chart of a method for reconstructing a three-dimensional building using a radar point cloud according to an embodiment of the present invention.

[0033] Figure 2 It is a schematic diagram of image point cloud timestamp alignment implemented by the present invention.

[0034] Figure 3 It is a rendering of the model reconstructed by the present invention. DETAILED DESCRIPTION

[0035] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present invention.

[0036] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0037] like Figure 1 As shown, the present invention comprises the following steps:

[0038] S1: Using the visual offset effect of point cloud and image, the time stamps of point cloud and image are accurately matched, so as to obtain the mapping relationship between point cloud and image, so as to support the texture mapping of subsequent models; and extract the texture features of the image;

[0039] Specifically: First, a platform that can visualize point clouds and images simultaneously is built, and then it is adjusted to a program that can display point cloud sequences and image sequences so that they are aligned according to their respective timestamps, such as Figure 2 As shown. By observation, since the timestamps of the point cloud sequence and the timestamps of the highlight sequence are obviously misaligned, it is assumed that their difference is Δt. Therefore, the relationship between the timestamps of the point cloud and the image can be expressed as:

[0040] T cam =T lidar +Δt

[0041] Among them, T cam The timestamp of the camera shooting, T lidar The timestamp of the laser radar.

[0042] Since the camera's pose is calculated based on the radar's pose and the transformation matrix from the radar to the camera, when the camera's timestamp relative to the radar changes, the camera's pose will also change. Here, the radar's pose is interpolated to index the camera's associated radar pose from a more fine-grained timestamp. Finally, the camera's pose can be expressed as:

[0043] P cam =P lidar ×T l2c

[0044] min|T cam -T lidar -Δt|

[0045] Among them, P cam is the camera pose, P lidar is the radar pose, T l2c is the transformation matrix from radar to camera, and || is the absolute value. The selection of the radar-camera timestamp difference Δt depends on how long the texture features of the image are closest to the texture features of the radar point cloud with the longest time interval.

[0046] S2: Based on the pose information on the continuous time trajectory, the point cloud is spliced ​​and cropped to extract the complete building point cloud; then the reflectivity threshold screening mechanism is used to filter the complete building point cloud to reduce noise interference and obtain the processed building point cloud;

[0047] Specifically, the point cloud is subjected to octree downsampling and clipping operations. Here, the point cloud is clipped to leave only the central building. The step of filtering the point cloud according to the reflectivity threshold screening mechanism further includes:

[0048] I p <T

[0049] Among them, I p It represents the reflectivity value of a single point p in the point cloud, and T is the set reflectivity threshold.

[0050] S3: Use a noise-resistant density-based spatial clustering algorithm to perform semantic recognition on the processed building point cloud to obtain various building semantic features and distinguish the building semantic features, including doors, windows, roofs, etc.; then combine the building structure attributes and the recognized building semantic features to refine the facade structure of the processed building point cloud and obtain complete building facade information;

[0051] Specifically:

[0052] First, the RANSAC (Random Sampling Consensus) algorithm is used to extract the building facade and the ground plane from the initial point cloud, and the parameters of these two planes are retained.

[0053] Next, the DBSCAN algorithm is used to automatically discover and distinguish point cloud clusters in different density areas by setting appropriate neighborhood distance and density thresholds, effectively distinguishing different components of the building. The DBSCAN process is as follows:

[0054] P1,P2,...,P n = DBSCAN(P,ε,minPts)

[0055] Among them, P is the complete point cloud output, ε is the neighborhood size, minPts is the minimum number of points in the neighborhood required to define a point as a core point, P1,P2,...,P n The n point clouds are finally clustered.

[0056] After completing the point cloud clustering process, the implementation steps of the present invention involve projecting the optimized and integrated point cloud onto a two-dimensional building facade plane to construct a basic facade diagram of the building. Subsequently, by identifying and analyzing the significant vertical gaps between the levels in the facade diagram, different floor structures are systematically defined. For these defined floors, the inherent laws of the building facade, namely the repetitive patterns and equidistant characteristics, are used to further refine and segment the window elements. It is worth noting that this method specifically marks the bottom floor of the building as the entrance area, while the top floor corresponds to the roof structure, so as to ensure that the first and last floors have a clear semantic distinction. Finally, based on these refined structured information, the present invention generates a comprehensive JSON file, which comprehensively and exhaustively encodes the structural features and semantic labels of the entire building, providing highly standardized and information-based data support for subsequent analysis, simulation and visualization.

[0057] S4: Use the Blender modeling tool to convert the complete building facade information into a building information model (BIM), and then add texture features to the building information model based on the mapping relationship between the point cloud and the image, thereby generating a three-dimensional building model with real texture. Specifically, based on the mapping relationship between the point cloud and the image, the texture features of the image are attached to the corresponding surface of the BIM model by image projection, and then the realism of the model is enhanced by adjusting the lighting model and material, thereby generating a three-dimensional building model with real texture, such as Figure 3 shown.

[0058] In texture mapping, since the number of images involved is too large, a part of the images are randomly selected for texture mapping. The image pose involved in texture mapping here uses the image pose optimized in step S1.

[0059] The present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of a method for reconstructing a three-dimensional building using a radar point cloud are implemented.

[0060] The present invention provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for reconstructing a three-dimensional building using a radar point cloud are implemented.

[0061] The present invention realizes the application of generating three-dimensional building models directly from point clouds. It has both real-time processing capabilities and innovative processing strategies in operation. It is widely applicable to multiple fields such as architectural design, urban planning, and cultural heritage protection, and provides a new approach for three-dimensional building reconstruction that is both efficient and accurate.

[0062] Finally, it should be noted that the above embodiments and explanations are only used to illustrate the technical solution of the present invention rather than to limit it. Those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope disclosed in the technical solution of the present invention, which should be included in the scope of protection of the claims of the present invention.

Claims

1. A method for reconstructing a three-dimensional building using radar point cloud, characterized in that: The following steps are involved: S1: Using the offset effect of point cloud and image, the time stamps of point cloud and image are matched to obtain the mapping relationship between point cloud and image; the texture features of image are also extracted; S2: Based on the pose information on the continuous time trajectory, the point cloud is spliced ​​and cropped to extract the complete building point cloud; then the reflectivity threshold screening mechanism is used to filter the complete building point cloud to obtain the processed building point cloud; S3: Use a noise-resistant density-based spatial clustering algorithm to perform semantic recognition on the processed building point cloud to obtain various building semantic features; then combine the building structure attributes and the recognized building semantic features to refine the facade structure of the processed building point cloud to obtain complete building facade information; S4: The complete building facade information is converted into a building information model, and then texture features are added to the building information model according to the mapping relationship between the point cloud and the image, thereby generating a three-dimensional building model with real texture.

2. The method for reconstructing a three-dimensional building using radar point cloud according to claim 1, characterized in that: In S1, feature points in the image and feature points in the point cloud are identified, the feature points in the image are matched with corresponding features in the point cloud, and then the mapping relationship between the point cloud and the image is obtained after minimizing the spatiotemporal deviation between the two through an optimization algorithm.

3. The method for reconstructing a three-dimensional building using radar point cloud according to claim 1, characterized in that: In S2, the complete building point cloud is filtered using a reflectivity threshold screening mechanism to obtain a processed building point cloud, including: The reflectivity value I in the complete building point cloud p Point clouds p with reflectivity less than the reflectivity threshold T are removed.

4. The method for reconstructing a three-dimensional building using radar point cloud according to claim 1, characterized in that: In S3, the facade structure of the processed building point cloud is refined by combining the building structure attributes and the recognized building semantic features to obtain complete building facade information, specifically: According to the feature repetition and equal spacing of architectural semantic features in the building structure, the feature missing parts and erroneous features in the facade structure of the processed building point cloud are fine-tuned to obtain complete building facade information.

5. The method for reconstructing a three-dimensional building using radar point cloud according to claim 1, characterized in that: In S4, the complete building facade information is converted into a building information model using a Blender modeling tool.

6. The method for reconstructing a three-dimensional building using radar point cloud according to claim 1, characterized in that: In S4, according to the mapping relationship between the point cloud and the image, the texture features of the image are fitted to the corresponding surface of the BIM model by means of image projection, and then the realism of the model is enhanced by adjusting the lighting model and the material, thereby generating a three-dimensional building model with real texture.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Citation Information

Patent Citations

  • Building template instantiation modeling method based on point cloud data

    CN115033967A

  • Pipeline BIM model reconstruction method and system based on depth image and point cloud

    CN115294294A

  • Map element reconstruction method and device, computer equipment and storage medium

    CN115830073A

  • Automatic point cloud splicing method based on multiple radar devices

    CN116721239A

  • Modeling method and device based on three-dimensional GIS and BIM integration

    CN118262062A